If no card is revealed, most likely because that player's library is empty, the exploring creature receives a +1/+1 counter.
Spyglass Siren
In some unusual cases, noncreature permanents may explore. For example, if the creature card returned by Defossilize is somehow not a creature once it's on the battlefield, it can still explore. You'll take all the same actions, and you may end up putting a +1/+1 counter on the permanent. (Note that some effects target a creature, and those effects would still require a legal target to have it explore.)
If an ability instructs a creature to explore, its controller reveals the top card of their library. If it's a land card, they'll put it into their hand. Otherwise, they'll put a +1/+1 counter on that creature, then choose to either leave that card on top of their library or put it into their graveyard.
Magic: The Gathering · ™ & © Wizards of the Coast · Illus. David Astruga
Card data via Scryfall. Rarix is fan-made; not endorsed by the rights holders.
