1

I am looking for the right AI tool to interpolate between binary images to create an animation. The images show segmented cells of a certain plant structure. The input would be start and end point images of the growth/evolution of the structure and the output would be a series of possible transition images.

These are meant for illustration purposes only, there is no criterion for truth as the transitions are unknown and these won't even be treated as hypotheses. Ideally, "realism" would be achieved by the AI seeing individual blobs in the image as cells and re-arranging/dividing/re-shaping/eliminating them accordingly to achieve the transitions.

For example, the transition between the two images below would have to add cells and make more lobes in some plausible way.

What would be the right neural network(s) to try on this? Is there something that might be able to do it out of the box or would one have to try train it in some way? It seems like if we can make stable diffusion animations from only the text prompts for the first & last images, this task is much simpler, but may remain unexplored because it's so specific.

Segmented Asteroxylon fossil

Segmented Hestia fossil

Improve this question
2
  • Would the input simply be the images, or would you also manually select several key points and mark where they appear on each image?
    – Ray Butterworth
    Oct 10, 2022 at 14:40
  • I could select key points. There will often be transitions that don't have a 1:1 mapping, though (e.g., number of vertices of shape outline changes, number of cells covering shape changes, etc.).
    – Martin Bouda
    Oct 11, 2022 at 12:41

1 Answer 1

Reset to default
0

With just those two images, as you said, no correspondence to any reality. You might not even get acceptable results for illustration purposes. Consider having more inbetween stages?

I would look for (first) an algorithm/software that morphs the outer boundaries of one set to the other set (and I would just specify that outer boundary manually instead of trying to compute it). Actually, you want to morph the entire area of the first set to the second set. But if you have a boundary morph you can propagate that to the interior area as well by using conformal mapping. Once you have an area mapping, use that to map the individual cells in one to the other via a greedy nearest neighbor algorithm.

OK, I know you asked for software not an algorithm. But your search will be enhanced if you search using the terms in my description above.

Improve this answer
1
  • Thanks, I've actually started working on an algorithm roughly along these lines in the mean time. It's good to see someone else thinks it may be a good idea and thanks for the pointers.
    – Martin Bouda
    Oct 11, 2022 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.