News from the lab

I guess this post is as good a place as any to announce that I recently got tenure. The reason why this post starts with a humble (?) brag is that it is customary in Germany to give an inaugural lecture (Antrittsvorlesung) upon tenure. These lectures tend to be less scientific in nature, and so I decided to reflect on where computational microbiology would be heading into the future. Obviously, it was not possible for me not to include a mention of AI, especially given the acceleration driven by coding agents since the fall of 2025. It ended up being a bit of a gloomy outlook for young people looking to study computational biology; what was once a sure path to a stable (?) career is now under threat of being quickly automated. Anthropic’s BioMysteryBench benchmark has shown how AI agents can solve complex bioinformatics tasks, and I like to show skeptics the plots on ARC-AGI’s website, which not only report how many tasks are solved by AI, but also at which cost, which is obviously a fraction of that of a human, hence the gloomy end of my inaugural lecture.

Let’s however imagine, as AI enthusiasts do, that AI-assisted work simply increases productivity without necessarily removing a bioinformatician’s job. How useful, costly, and safe is it to use AI agents for scientific work? With this post I want to add a single data point to the discussion, and look in some more detail at various implementation details that are specific to the German sensitivity to data protection (Datenschutz) and independence from foreign companies (broadly gestures at everything).

The task

I gave myself the task of collecting as many large bacterial datasets as possible that were used for GWAS/ML studies, and in particular of focusing on those in which the genetic variants causal for a phenotype were experimentally validated. The final output I wanted was a table with as many details as possible. A boring chore that seems perfect for an agent to do. I therefore prepared an initial set of 15 papers (identified through the excellent Google Scholar Labs tool), for two of which I had already filled the table with some sensible fields. The papers were provided as a Zotero export in PDF format. I wanted the agent first to fill the table with details found in the papers, and then to fetch and parse additional ones.

The models and their harnesses

I wanted to test whether this relatively simple but open-ended task could be performed well by a set of AI models ranging from the frontier to the “portable” (i.e. locally hosted) ones. I therefore chose the following, in order of “size”:

• Claude Opus 4.8 (using an Anthropic personal Pro license)
• glm-5.2 (via a pay-as-you-go OpenRouter API key)
• qwen-3.5-397b-a17b (via an Academic Cloud/GWDG API key, free, but capped at 2000 requests per day)
• qwen-3.6-35b-a3b at 4-bit quantization (running on our lab’s workstation with two Nvidia 3090 GPUs)

The use of the first two models clearly comes with data and intellectual property protection issues, while with the other two we can ensure that data does not leave Germany and is not stored and used for further training.

In terms of providing a harness to these models to become actual agents (as per the “official” definition), I used Claude Code for Claude, and the pi agent harness for the other three, with two extensions to talk to the Academic Cloud models and to host the smaller model locally. To avoid any collateral damage to files outside the working directory, I also used a sandboxing script based on the bwrap tool. Lastly, I provided two skills to all agents: to use the excellent liteparse tool to extract text from PDFs and to use uv to set up isolated Python environments.

The prompts

I started with this prompt to fill the table with details from the initial set of 15 papers:

I want to complete the table in datasets.tsv so that the information about bacterial GWAS datasets contained in this repository is complete. I am particularly interested in identifying which of these studies have an experimentally validated hit (gene presence/absence, short variants, or other kinds of variants). Parse the papers using liteparse as needed and fill any missing information in the table and add the missing entries. If you need to go online to find the source data/code from each paper feel free to do so. If you think more columns should be added to make the table more detailed please do so. Also prepare a markdown file detailing the steps you have taken and any trouble you have faced.

Once that was done I asked the agents to fetch additional papers:

Now try to add more entries to the table: query pubmed using their entrez API (you can for instance use the Bio.Entrez module from Biopython, use uv to set the environment, and use my email XX.YY@twincore.de, be careful not to exceed their quota). Find relevant papers listing large bacterial GWAS datasets, focusing (but not exclusively) on those with experimentally validated hits. Make a list of PMIDs, I will then download them and provide them for parsing and completing the table. Continue to document what you are doing.

I then created a Zotero folder from the list of PubMed IDs provided by the agent, got the PDFs, and exported them as a new Zotero folder. This was the last instruction given:

I created a new Zotero folder called XXX. Complete the datasets.tsv file, adding a column to indicate that the new entries come from an AI-assisted search

Hiccups and cost

All four agents completed the task in a reasonable amount of time (no more than an hour, excluding the time I spent clicking around in the browser to fetch the PDFs Zotero could not automatically find). The only agent requiring intervention on my part was the one using the Academic Cloud model provider, which got too busy and started hanging; I downgraded the model to qwen-3.5-122b-a10b and could continue the task.

The only model for which I could measure inference costs directly was glm-5.2, which consumed $1.75 overall. I did not reach the usage limit on either my Anthropic personal Pro account or in Academic Cloud, for which I used 94 requests over the 2000 daily limit. The local agent likely increased the temperature in my office slightly, but luckily I was working from home that day. There’s a lot of hidden or subsidized costs in all four agents, but it’s clear how this task was executed on the cheap.

Results

More importantly, did the agent perform well? I was interested in comparing the way they filled out the table from the original set of papers, but more importantly if they identified more datasets that I had originally missed.

The most interesting result is that none of the additional 80 papers was identified by all four agents; a single one (a PhD dissertation!) was identified by three, and 13 by at least two agents. It is therefore also likely that running the same agent multiple times would have given different results. A truly open-ended task.

Another interesting, or rather concerning, result is that qwen-3.5-397b-a17b/122b-a10b missed the .bib file in the Zotero export folder and filled the table with hallucinated DOIs. This agent also had the weakest strategy to filter its own PubMed searches, including five papers that had nothing to do with bacteria. Hallucinations are still a thing, even in an agent loop.

Surprisingly, the smallest (and quantized!) model performed well, suggesting only eight new datasets, but with explicit filters to identify bona-fide bacterial datasets with an actual experimental validation.

In the heavyweights league, the underdog glm-5.2 differentiated itself from the incumbent by not using a binary classification of whether a dataset had an experimentally validated hit, but added a third “partial” category to flag unclear examples.

The aftermath

After running this experiment I had Claude further refine the four tables to remove the smaller studies, fix the obvious hallucinations, and harmonize the columns. I then dumped the resulting table on poor Alessio for a round of manual curation. The final result turned out to be quite useful for some work we are doing in the lab.

I can see how my own experience and preference in tools influenced the agents; I could have simply given a prompt with no initial set of papers, no preferred tools (i.e. liteparse and biopython), and removed myself completely from the loop. I would imagine that an AI-native scientist would have let the agent figure out which tools and strategy to use, likely leading to a different set of datasets, but possibly making the identification of mistakes more difficult.

Despite the hallucinations and small sample size, this little experiment shows that agents can be useful for open-ended work, provided that multiple agents are run and that a human stays in the loop to check for gross errors. More importantly, locally hosted models seem to already be on par with their larger counterparts, at least when given a relatively narrow task. This is a very exciting possibility, especially for people working on sensitive data or worried about depending on foreign entities. The EU however still needs to produce more of its own open-weight models to be truly independent.

Last week, during a brutal heatwave, we had the great pleasure to have a new PhD graduation from our lab! Hien defended her thesis from the questions of her two examiners, Caroline Wendling and Marius Vital. Hien did a great job, and has a result she did not need to make any changes to her dissertation.

Fun fact: while Hien was defending her thesis, her long-running experiment in our Chi.Bio bioreactors was still running in a very warm lab. Such are the joys of automated laboratory evolution!

Congratulations to Hien!

We have just posted a new preprint, describing our work on improving how to detect bacterial transmission in hospitals using genomics. This effort was led by Judit, who worked in collaboration with colleagues from Hannover Medical School and Copenhagen University Hospital-Rigshospitalet, to use a large (~30k genomes!) bacterial genomics dataset that our collaborator Susanne Häußler has put together over the years.

Bacterial infections are a much too common “perk” for patients being hospitalized, and it’s the job of epidemiologists to identify the transmission routes for these pathogens. Often this problem boils down to a relatively simple question: is this particular pair of bacterial samples related to each other? Genomics, used to read all the millions of letters in the bug’s DNA, provides the highest possible resolution to answer this question. But how to choose the number of genetic differences (“SNPs”) that separate related (i.e. the same bug moving between patients) from unrelated samples?

The standard practice in the field has relied on a fixed SNPs threshold (e.g. 20), which generally works, but has two problems: it’s rather arbitrary, but most importantly it does not take into account the impact of time. Since every time a bacterial cell duplicates there is a chance that errors (i.e. SNPs) are introduced, then samples that are farther apart in time can be expected to have more SNPs separating them. But how can we calibrate such a “SNPs accumulation clock”? Judit had the brilliant intuition that in the dataset we had ~50 patients that had been sampled multiple times (~20!) over their hospital stay. She could then calibrate our empirical clock within the same dataset we would use the clock for. We hope that this approach will be taken up by genomic epidemiologists in their daily practice.

Once we had used our calibrated clocks to identify transmitting bugs, we wanted to know if we could identify genetic characteristics that could differentiate them from non-transmitting ones. We used two approaches to answer this question, one using lists of known genes, and one looking at the whole “haystack”. Even though we could identify many genes and genetic variants associated with the ability to transmit between patients, we failed to use them to predict which samples were part of a transmission chain in a held-out dataset. This suggests that patient and environment factors might dominate the probability of bacterial transmission. Measuring and including these factors in future analysis may then lead to a system that is better for predictions.

There’s more to discover in the preprint and the accompanying code repository, so please dig in!

We are very happy to report that the fifth (!) PhD thesis from the lab has been submitted this month!

As it is customary with many graduate students, Hien submitted her thesis one day before the deadline; but this time I am to blame for pushing her to finish one last experiment before completing her thesis.

We are all very excited about her hard work and are now looking forward to the public defenses in June.

Congratulations to her!

Update (2025-12-16): the position has been provisionally filled. Thanks to all who applied!

After a rather long hiatus we are hiring again!

We are looking for a candidate to fill a computational PhD student position, with the main task of better understanding which genetic elements (“genes”) make bacterial pathogens such as E. coli, K. pneumoniae, P. aeruginosa, and E. faecium virulent and resistant to antibiotics. We have in fact developed methods (here and here) to sift through large numbers of bacterial genomes for this exact purpose (see here, here, and here), and it will be the job of the candidate to use and improve these methods so that they can be applied to an even larger number of genomes. We are also particularly interested in implementing more machine learning methods and to integrate molecular phenotypes such as gene expression and proteomics.

Our lab is part of the RESIST excellence cluster at Hannover Medical School (MHH), and we are part of a collaborative project with Susanne Häussler and Meike Stiesch to identify genetic determinants of pathogenicity and virulence in life-threatening bacterial infections.

We are looking for a candidate with a strong computational background and relevant technical skills:

• Strong background in computational biology, bioinformatics, computer science, or a related field.
• Experience with programming languages such as Python and workflow management systems such as Snakemake.
• Familiarity with machine learning techniques and libraries (e.g., scikit-learn, TensorFlow, PyTorch).
• Knowledge of bacterial genomics and related bioinformatic tools.
• Experience with high-performance computing (HPC).
• Experience with version control systems (e.g., git) and software development best practices.
• Experience with statistical analysis and data visualization.

We offer a fully-funded PhD position for a little longer than 3 years, to begin in February 2026. The student will be enrolled in the Biomedas graduate program, which offers curriculum specifically designed for computational biology students. The lab is located at Twincore, embedded in a large scientific campus that include Hannover Medical School, and the Center for Individualized Infection Medicine (CiiM); we are also part of the Helmholtz Centre for Infection Research (HZI), which offers lots of opportunities for collaboration.

Hannover is the capital of Lower Saxony, in northern Germany, and is an affordable and vibrant city.

Please apply as soon as possible by sending the following documents to Marco Galardini at marco.galardini@twincore.de:

1. Your CV
2. A short motivation letter
3. The link to a software portfolio (e.g., your GitHub account)
4. Contact details of two references