

I tried throwing this image at qwen3.6 locally with the prompt “What is in this image?” and it spat out:
Based on the visual evidence, this is a micrograph (a photograph taken through a microscope) of bacteria.
Here is a breakdown of what you are seeing:
- Organism Type: These are bacteria. You can tell by their small size relative to the scale bar in the bottom right corner (which represents 10 micrometers).
- Stain: They are stained purple/blue, which indicates they are Gram-positive bacteria. In a Gram stain, Gram-positive bacteria retain the crystal violet dye and appear purple.
- Shape: The individual bacterial cells are spherical or round, which classifies them as cocci (singular: coccus).
- Arrangement: The most distinct feature is that the cocci are arranged in chains.
Conclusion: This image shows Gram-positive cocci in chains, which is the characteristic appearance of bacteria from the genus Streptococcus (or possibly Enterococcus). Common examples include Streptococcus pyogenes (which causes strep throat).
So you might be able to get further than you expect. You might try collecting some of your own sample data for things you understand (like particular bacteria you’re interested in) and providing those images along with a prompt that includes labels for your sample imagery. (Like “Image 1 is an example of (…whatever…). Image 2 is an example of (…something else…). Which (if any) of these are in Image 3?”)
I don’t know how well it would work, but worth a try.
The dream, for me, is to get it to understand the protein structure files and DNA sequence files
Even then, much of what I’d want help with looking at proteins is how to write analyses of points in 3D space
I tried throwing this image at qwen3.6 as well, and it spits this out (but takes a REALLY long time to do it on my local hardware):
Based on the visual evidence, this image is a 3D molecular visualization of a protein structure, likely created using software such as PyMOL, UCSF Chimera, or VESTA.
Here is a breakdown of the specific components visible in the image:
The Protein (Blue Ribbon):
- The large, coiling blue structure is the protein backbone.
- It is rendered as a “ribbon” or “cartoon” diagram, which is a standard way to visualize the secondary structure of proteins.
- The coiled sections represent alpha-helices, which make up the majority of this protein’s structure. This “helix bundle” fold is characteristic of proteins in the globin family, such as myoglobin or hemoglobin, which are responsible for oxygen transport and storage.
The Central Ligand/Co-factor (Middle Right):
- Nestled within the protein fold is a complex molecule shown in a “stick” and “sphere” representation.
- The Flat Ring: You can see a flat, ring-like structure with blue sticks (representing nitrogen atoms). This is characteristic of a porphyrin ring.
- The Red Sphere: There is a large red sphere in the center of the ring. In this context, this almost certainly represents an Iron (Fe) atom.
- Conclusion: This complex is a heme group (or heme prosthetic group). Heme is the molecule responsible for binding oxygen in hemoglobin and myoglobin, or for electron transfer in cytochromes. The long chains extending from the ring are likely the side chains of the heme molecule (propionate groups) or a bound drug/inhibitor interacting with the iron.
The Ion (Bottom Right):
- There is a small molecule floating near the bottom right of the protein.
- It has a yellow center with four red arms arranged in a cross/tetrahedral shape.
- In chemical coloring (CPK coloring), yellow represents Sulfur and red represents Oxygen. This indicates a Sulfate ion ($SO_4^{2-}$). Sulfate ions are common in crystal structures of proteins and are often used to stabilize the structure or mimic phosphate groups.
Summary: You are looking at a structural biology model of a protein (likely a globin like myoglobin) containing a heme cofactor (the molecule with the iron) and a sulfate ion. This type of image is used by scientists to understand how the protein works at the atomic level.
You might be able to it to do something useful with analysis of protein renderings, but I think you’ll need some creativity with the prompts to stop it from overthinking things… It was able to recognize this as myoglobin in the “thinking” stage quickly enough, but then kept second guessing itself about various aspects for a long time. I also tried throwing it at gemma4 but it didn’t really give me much besides recognizing it as some sort of protein structure.
hook it up to some lab robotics and automate experiments
You might find this project and associated HN post interesting – it’s basically someone doing that with electronics analysis instead of biology, IIUC. No idea how well it works, but interesting to look at and think about.
Maybe there’s a coding community on lemmy that’s a decent place to talk shop on how to build these scripts up and what local models are good at what?
There’s !localllama@sh.itjust.works and probably some communities on lemmy.dbzer0.com as well – they’re not super-active though (at least from what’s made it to my attention browsing by all).














Wasn’t seeing it here on reddthat yet until I tried loading it manually, so I’m guessing the bot that automatically subscribes to new communities failed.