Process engineers at CDMOs do not lack for data. A single 200L fed-batch run generates tens of thousands of time-series points from DeltaV or a PI historian, plus MES batch records, operator logbook entries, and QA deviation notes. The Benchling ELN sits in the middle of all of this, expected to be the canonical record of what happened. In our experience, the hard part is not connecting the systems. It is figuring out what that connection should actually look like once you get there.
The Result Registration Schema Problem
When a fermentation run completes, someone has to register the results in the ELN. Sounds simple. In practice, a CDMO team we worked with tracked this step across 60 consecutive runs and found the average registration lag was 4.2 hours post-harvest. Not because engineers were slow. Because there was no agreed schema for what "results" meant in Benchling when half the data lived in PI and the other half was in a DeltaV batch summary PDF.
The schema question is boring. It is also the thing that kills most integration projects before they ship. You need to decide upfront: is the ELN the system of record, or is it receiving a summary from the MES? These are not the same thing. If Benchling is the system of record, your PI Web API pipeline needs to write structured result fields into Benchling entities, not dump a CSV attachment into a notebook. If Benchling is receiving a summary, the entity linking still matters but the read direction flips.
Our recommendation: treat Benchling as the scientist-facing layer and the historian as the raw data layer. Your result schema in Benchling should capture the process outcomes that inform next-run decisions, not every logged parameter. For a typical microbial fermentation, that is: final titer, viable cell density at harvest, total oxygen uptake, peak specific growth rate, and any deviation events with impact classifications. That is five to eight fields, not five hundred.
Entity Linking Between ELN and DeltaV/PI Batches
Here is the thing about entity linking: it only works if your batch IDs are consistent across systems from the start. DeltaV creates batch records using its own naming convention. PI inherits that batch ID as a tag prefix. Benchling has its own entry and notebook structure. Unless someone enforced a shared batch identifier at program setup, you are now doing forensic ID matching after the fact.
We have seen three patterns in practice:
- Manual field on the Benchling entry. Engineer types the DeltaV batch ID into a custom field when creating the notebook entry. Fragile. Typos cause orphaned records. But it is what most sites start with.
- MES-triggered notebook creation. The DeltaV batch start event (via OPCDA or the PI event frame) fires a webhook that auto-creates a Benchling entry with the correct batch ID pre-populated. This is the right architecture but requires infrastructure work up front.
- Post-run reconciliation via API. A nightly job queries the PI batch database, looks for entries where the Benchling custom field matches, and back-fills any missing links. Slower, but it catches errors the MES-triggered approach misses when the batch gets renamed mid-run.
In practice, most CDMOs run a hybrid of patterns one and three, with pattern two as a roadmap item. That is honest. Start with what you can deploy this quarter.
Automated Data Pull: API Rate Limits and the PI Web API Pipeline
PI Web API is powerful and poorly documented on rate limits. Officially, OSIsoft (now AVEVA) does not publish a hard cap, but in our testing against PI servers running on typical CDMO infrastructure, sustained queries above 50 requests per second reliably caused timeouts or degraded the historian's real-time write performance. That matters on an active bioreactor floor.
The safe pattern is batch-interval pulling, not streaming. For a completed run, pull the full time series in one request per tag per batch using the recorded-data endpoint, not a series of smaller windowed queries. For active runs, poll at no more than one-minute intervals for the tags that feed your alert model. Aggressive polling of an OSIsoft historian is one of the fastest ways to generate a hostile conversation with your automation team.
On the Benchling side, the API rate limit is 429-based and documented: 300 requests per minute for most endpoints at the time of writing. The main trap is notebook entry updates. If you are writing structured result fields into a Benchling entry, batch your writes into a single PATCH call rather than sending one field at a time. We have seen integrations that issued 40+ API calls to update one entry. One call should do it.
The pipeline architecture that works in practice: PI Web API pulls go into a local processing layer (a containerized Python service works fine). That layer computes derived values (DO area under curve, pH excursion minutes, etc.), then writes a single structured payload to Benchling. No direct PI-to-Benchling calls. The intermediate layer is where you do validation, unit conversion, and exception handling.
Deviation Annotations in Notebook Entries
Deviation documentation is where ELN integration gets genuinely complex, because now you are connecting the scientific record to the quality system. Two failure modes we see constantly:
Failure mode one: The deviation is documented in the QMS (a deviation report in Veeva Vault or MasterControl) but the ELN notebook entry for that run has no reference to it. The scientist writing the next-campaign report has to manually cross-reference two systems. This happens in roughly 60% of CDMOs we have assessed, by self-report.
Failure mode two: The deviation is annotated in the ELN but with unstructured free text. "pO2 dropped around hour 14, cascade kicked in, recovered." That is not queryable. When you want to retrospectively analyze all runs where DO fell below 20% saturation for more than 15 minutes, the annotation is useless unless it was tagged with structured fields (deviation type, affected parameter, time window, impact classification).
The fix is a structured deviation entry schema in Benchling with a mandatory QMS cross-reference field. Not glamorous. Needs buy-in from QA. Worth the political effort.
Common Integration Pitfalls
After working through several of these projects, certain failure patterns repeat. Reliably.
Pitfall: treating the integration as an IT project. Process informatics integrations that are owned entirely by IT, without a process engineer or QA SME in the room, consistently produce technically functional systems that scientists do not use. The schema choices, the field names, the deviation classification taxonomy, all of these need input from the people who will be entering data.
Pitfall: underestimating the historian data quality problem. PI and DeltaV historian data looks clean in the trend viewer. It is often not clean when you pull it programmatically. Compressed data, gap-filled values, instrument calibration periods that left artifacts, batch tags with inconsistent engineering units across campaigns. Plan for a data cleaning layer. Plan for it to take longer than you expect.
Pitfall: ignoring the SciNote path. Not every CDMO uses Benchling. SciNote is common at smaller shops and has a REST API that follows similar patterns. If your client uses SciNote, the architectural approach described here still applies, but the entity model is different. Do not assume Benchling-specific field names will map cleanly.
Real talk: the biggest pitfall is attempting the full integration in one project. We have seen six-month integration programs stall completely because scope crept to include every data source in the building. Start with one program, one bioreactor line, and three result fields. Ship that. Prove the data flows correctly through a full campaign. Then expand.
What This Looks Like When It Works
When the integration is functioning, the ELN becomes the place where a process engineer sees both the run outcome and the deviation history, cross-referenced and queryable. A QA reviewer can open a Benchling entry for a batch under disposition review and see the structured result fields, the PI-sourced time-series summary, and the deviation annotations linked to their QMS counterparts. No separate dashboard. No context switching.
That is the actual goal. Not system connectivity for its own sake. One place where the scientific record is complete.
Fermentile's integration layer is designed to handle the PI Web API and Benchling API connections described here, with a pre-built result schema for fermentation runs that maps to common Benchling entity templates. If you are evaluating how to structure this for your CDMO, we are glad to walk through the architecture. Request a demo and we can start with your historian topology.
