Cover the baby potatoes with cold water. Bring to a boil and simmer over medium heat until potatoes are firm but easily pierced with a fork, about 25 minutes.
Let cool slightly. Cut off a small slice from both ends so that the potato sits upright on the flat surface.
Top with a dollop of creme fraiche, a smaller dollop of caviar and a chive.
Sprinkle with salt and a few drops of fresh lemon juice. Enjoy immediately.