We all know there’s a lot of hype and skepticism around AI, and over the last year or so I’ve been hearing a lot about “Agentic” AI. I’ve struggled to get a real grasp on what that means without working examples; however, I’ve began to see hints of something. Videos mocking coders who are scrolling their phones while waiting for the AI to complete a task. Peers claiming Claude but not GPT can do complex reasoning and planning. Not much, but enough for me to stop ignoring the term as purely buzz word.

Agentic AI is defined as “an autonomous systems that act independently to achieve complex, multi-step goals without continuous human oversight.” This seems fanciful, but my basic understanding is that these Agentic systems are do the large scale reasoning then use other apps to achieve smaller sub-goals. Essentially these systems allow for pipelines to be set up as verbal lists of tasks then they work their way through the tasks with some perhaps limited problem solving. A crucial aspect of this seems to be that if you give the bot more tools it can do more and handle more failures. Sometimes more tools means a text book or document on your work to help it reason and plan. Sometimes more tools means writing a script for it to use in future analyses.

Now, while these sound mildly interesting, they’re essentially useless if they’re locked behind a pay wall. I’m not paying some company to think poorly for me. Someone else’s tools are not an extension of my skills or personal power since I’d be neither able nor willing to build on them. However, the notion of Local Agentic AI changes this. If it’s on my computer even if I don’t fully understand what it’s doing, I can build on it. I can control it and treat it as an extension of myself – as humans do with all tools.

I’m a modest coder, and even the basic AI has expanded my abilities there just by helping me find algorithms I wouldn’t have known how to find before. I have ran Local LLMs, but I’ve not tried these Agentic LLMs. I worry I was unimpressed too quickly, and gave up on a potentially useful tool. If I can tell the local agent to make a rough version of a function that does XXXX, then I can get more done. If I can tell it to write a simple script that makes this table that I’d normally just do by hand, check the script, then link that scipt to a command for the task I wouldn’t normally trust the AI with then the AI can do a larger chunk of my work. The more scripts I make, the more the AI can do. The more scripts I download from open source communities, the more the AI can do. I don’t have to trust the AI if all it’s doing is moving information around and triggering scripts. I just have to check the scripts. If we start adding in robotics… yeah, I can see the hype.

Of-course, the counter argument is that we’ve had IFTTT triggers and pipelines for decades. So maybe this isn’t fundamentally new, but is it still an impetus to download more tools and build more pipelines? Will I fall behind if I don’t figure out how to use this efficiently and effectively (FOMO)? Does anyone here have experience with Agentic LLMs (especially local)? Also, what’s the best Lemmy community for learning more about this sort of thing and maybe also hooking it up to basic robots?

  • tristynalxander@mander.xyzOPEnglish
    2·
    1 day ago

    I think it’s very much a “how much data on this exist” sort of problem for most of these. Like I can pick out bacteria from fungi on an augur plate trivially, but I don’t know if there are databases of augur plates characterizing different growths with different background colors and all the diversity that real life has. Honestly, I haven’t tried this yet. It might be able to get it just fine or might be able to get it if I backlight the plate - of course, at that stage there are other programs for detecting colonies.

    The dream, for me, is to get it to understand the protein structure files and DNA sequence files then hook it up to some lab robotics and automate experiments that are mostly trivial but slightly dynamic. Maybe start with something simple like cloning then build out to other methods. Some of this stuff exists already but companies charge you a fortune and go out of business (or get bought up and discontinued) constantly, so it kinda needs to be stuff I can build and maintain myself – or FOSS.

    Even for purely computer stuff, anytime I try to get the AI to help with my proteins, it’s functionally useless because it doesn’t have a way to “see” the protein’s structure file. I can write my own scripts to help with that, but I’ll have to work on the connection between the language the AI thinks in and the actual things my code detects. Or maybe I can tell it to ask questions based on the writing then run code that analyzes the protein to give answers to specific questions… Even then, much of what I’d want help with looking at proteins is how to write analyses of points in 3D space, and while it has helped me pick the right algorithms (sometimes), I’m haven’t really been able to give it enough information to let it check that things are being implemented correctly (I think this is alignment). That might something like hooking it up to pymol (3D viewer) or it might just be a bit too dumb. It’s hard to say without trying it, and it’s a lot of work for something it’s likely to get get confused about even with the ability to “look” at the protein structure.

    I feel like, for coding, one thing I’m going to have to get it to do is stop after it makes a function or something so I can check that it’s still going where it’s supposed to go or tell it what the next function needs to do. I don’t know. Maybe I’ll start with lots of hand holding then slowly build it up until it can reliably do more or I can’t get it to be reliable enough. Maybe there’s a coding community on lemmy that’s a decent place to talk shop on how to build these scripts up and what local models are good at what?

    • e0qdk@reddthat.comEnglish
      2·
      1 day ago

      I tried throwing this image at qwen3.6 locally with the prompt “What is in this image?” and it spat out:

      Based on the visual evidence, this is a micrograph (a photograph taken through a microscope) of bacteria.

      Here is a breakdown of what you are seeing:

      • Organism Type: These are bacteria. You can tell by their small size relative to the scale bar in the bottom right corner (which represents 10 micrometers).
      • Stain: They are stained purple/blue, which indicates they are Gram-positive bacteria. In a Gram stain, Gram-positive bacteria retain the crystal violet dye and appear purple.
      • Shape: The individual bacterial cells are spherical or round, which classifies them as cocci (singular: coccus).
      • Arrangement: The most distinct feature is that the cocci are arranged in chains.

      Conclusion: This image shows Gram-positive cocci in chains, which is the characteristic appearance of bacteria from the genus Streptococcus (or possibly Enterococcus). Common examples include Streptococcus pyogenes (which causes strep throat).

      So you might be able to get further than you expect. You might try collecting some of your own sample data for things you understand (like particular bacteria you’re interested in) and providing those images along with a prompt that includes labels for your sample imagery. (Like “Image 1 is an example of (…whatever…). Image 2 is an example of (…something else…). Which (if any) of these are in Image 3?”)

      I don’t know how well it would work, but worth a try.

      The dream, for me, is to get it to understand the protein structure files and DNA sequence files

      Even then, much of what I’d want help with looking at proteins is how to write analyses of points in 3D space

      I tried throwing this image at qwen3.6 as well, and it spits this out (but takes a REALLY long time to do it on my local hardware):

      Based on the visual evidence, this image is a 3D molecular visualization of a protein structure, likely created using software such as PyMOL, UCSF Chimera, or VESTA.

      Here is a breakdown of the specific components visible in the image:

      1. The Protein (Blue Ribbon):

        • The large, coiling blue structure is the protein backbone.
        • It is rendered as a “ribbon” or “cartoon” diagram, which is a standard way to visualize the secondary structure of proteins.
        • The coiled sections represent alpha-helices, which make up the majority of this protein’s structure. This “helix bundle” fold is characteristic of proteins in the globin family, such as myoglobin or hemoglobin, which are responsible for oxygen transport and storage.

      2. The Central Ligand/Co-factor (Middle Right):

        • Nestled within the protein fold is a complex molecule shown in a “stick” and “sphere” representation.
        • The Flat Ring: You can see a flat, ring-like structure with blue sticks (representing nitrogen atoms). This is characteristic of a porphyrin ring.
        • The Red Sphere: There is a large red sphere in the center of the ring. In this context, this almost certainly represents an Iron (Fe) atom.
        • Conclusion: This complex is a heme group (or heme prosthetic group). Heme is the molecule responsible for binding oxygen in hemoglobin and myoglobin, or for electron transfer in cytochromes. The long chains extending from the ring are likely the side chains of the heme molecule (propionate groups) or a bound drug/inhibitor interacting with the iron.

      3. The Ion (Bottom Right):

        • There is a small molecule floating near the bottom right of the protein.
        • It has a yellow center with four red arms arranged in a cross/tetrahedral shape.
        • In chemical coloring (CPK coloring), yellow represents Sulfur and red represents Oxygen. This indicates a Sulfate ion ($SO_4^{2-}$). Sulfate ions are common in crystal structures of proteins and are often used to stabilize the structure or mimic phosphate groups.

      Summary: You are looking at a structural biology model of a protein (likely a globin like myoglobin) containing a heme cofactor (the molecule with the iron) and a sulfate ion. This type of image is used by scientists to understand how the protein works at the atomic level.

      You might be able to it to do something useful with analysis of protein renderings, but I think you’ll need some creativity with the prompts to stop it from overthinking things… It was able to recognize this as myoglobin in the “thinking” stage quickly enough, but then kept second guessing itself about various aspects for a long time. I also tried throwing it at gemma4 but it didn’t really give me much besides recognizing it as some sort of protein structure.

      hook it up to some lab robotics and automate experiments

      You might find this project and associated HN post interesting – it’s basically someone doing that with electronics analysis instead of biology, IIUC. No idea how well it works, but interesting to look at and think about.

      Maybe there’s a coding community on lemmy that’s a decent place to talk shop on how to build these scripts up and what local models are good at what?

      There’s !localllama@sh.itjust.works and probably some communities on lemmy.dbzer0.com as well – they’re not super-active though (at least from what’s made it to my attention browsing by all).

      • tristynalxander@mander.xyzOPEnglish
        2·
        1 day ago

        Yeah, I think this is conversation is entering the just try to build it territory, but definitely worth a try.

        There’s !localllama@sh.itjust.works

        Thanks. I’ll have to get through my current project, but this will be a good resource when I do more on LLMs.