You need to create a custom component and then one way would be to use eventHandlers to handle mouseover and mouseout events in combination with the marker ref to call leaflet's native openPopup and closePopup methods respectively.

const RenderIcons = () => {
  const markerRef = useRef();

  const eventHandlers = useMemo(
    () => ({
      mouseover() {
        if (markerRef) markerRef.current.openPopup();
      },
      mouseout() {
        if (markerRef) markerRef.current.closePopup();
      }
    }),
    []
  );

  return (
    <Marker
      ref={markerRef}
      position={position}
      icon={icon}
      eventHandlers={eventHandlers}
    >
      <Popup>Hello</Popup>
    </Marker>
  );
}; 

Demo

Using React Leaflet v3, the solution is easier and cleaner, use Tooltip instead of Popup, e.g.:

<Marker position={this.props.position}>
  <Tooltip>I appear on mouse over</Tooltip>
</Marker>

In particular, I have found useful to also add sticky property for Tooltip. There are more examples for tooltips in the documentation, which covers basic tooltips, sticky tooltips or permanent tooltips, with offsets and many more.

I think whatever you want to show as tooltip, just add that to the "title" of the div where you want to show it.

Eg:

<div title="I am the tooltip text">I am the div where you should hover</div>

But if its a custom designed div then go as the answers given before.