Will AI eat runtime workflows?
Critical runtime processes and integrations at major enterprises require things that fully agentic workflows can’t provide today. Low latency, high accuracy, auditability, economical executions.
But, as AI agents continuously get better at orchestrating work across systems, a sharp question has come up: will these processes eventually be swallowed by fully agentic workflows too?
To date, much of the enterprise adoption of AI has focused on three themes where the agentic pattern is already proving powerful:
- Create-time work like coding, design, research and copywriting.
- Customer-facing agents, such as voice and chat assistants that handle guest or customer interactions.
- Administrative orchestration, meaning AI connected to operational systems to streamline admin work.
Coding agents speed the development process tremendously, although the increased output puts a higher dependency on QA processes. Customer-facing agents armed with skills, knowledge, and context have proven to be an effective channel for customer support and even revenue generation. A business user orchestrating tasks across several systems, often through MCP connections to operational tools, is doing work that’s tedious, variable, and intermittent, which is exactly where agents shine.
Runtime is a different story. Given what tokens cost today, how reliable outputs are, and the latency of multi-step agent reasoning, fully agentic workflows are not yet an answer to complex enterprise integrations and runtime processes.
So where does AI fit within critical workflows? The preconditions today are that the mandate of AI within a process has to be tightly scoped, that it must be interoperable with existing systems, and that it has to align with how engineering teams already build software.
PolyAPI developed AI functions to support this pattern.
Architecture as leverage
The most powerful way to meet those preconditions is to write integrations and microservices using reusable primitives and native code. Following this principle means that a developer can leverage whatever coding agent they want to accelerate development, but still maintain consistency and quality. This architectural paradigm also ensures that enterprises do not create tight coupling between themselves and any vendor, whether that be an operational system, an agent harness, or a model provider. To read more about how Poly’s architecture gives enterprise leaders leverage, see What Makes Poly So Special.
In addition to offering the traditional primitives that support an enhanced full lifecycle development process, Poly developed AI functions so that steps within runtime workflows that rely heavily on inference, or that are very difficult to code, can simply be called as a function, backed by an AI model.
What are AI Functions?
A traditional function is a reusable piece of code with a predefined interface. It receives defined argument types, executes coded logic such as API calls, database queries, or queue interactions, and returns a defined output.
An AI function follows that same structure. Instead of hard-coded logic, it uses an AI model with a controlled prompt to perform the operation at runtime. You can configure it with tool calls that give the model selective access to systems and data. The Poly gateway injects credentials only when an approved tool call executes, so the model never sees them.
That interface is what makes them useful in production. Because an AI function looks like any other function, it can sit inside the same native-code workflow, be invoked through the same SDK, and be managed the same way as the rest of your integration components.
Before AI functions, a developer had to write and continually maintain custom logic for each variation, anticipating every usage scenario in advance. Now they configure an AI function and use it as an application component, without rewriting that logic every time requirements shift. Teams can automate more of the variable steps, deliver integrations faster, and adapt without rebuilding large portions of an application.
Calling a model API directly does not give you that. An API gives you access to a model. Operationalizing that model inside an application still requires architecture, controls, and security. AI functions supply them:
- Model independence. The application talks to the function, not to a specific model, so the architecture stays model-agnostic.
- Controlled scope. Context, tools, and data are defined in advance. The model sees only what the operation requires.
- Secure system access. Connecting a model directly to business systems means developers manage credentials themselves. Poly’s gateway injects credentials only when approved tool calls execute.
- Structure and reuse. Instructions, inputs, outputs, and tool access are packaged into a reusable component that can be managed and invoked across applications.
Where AI functions are most impactful
AI functions are strongest on the steps that are hard to express as fixed rules: unstructured or multimodal inputs, context-dependent inference, judgment-based scoring or ranking, and personalized content. They are the wrong choice for ultra-high volumes, large payloads, purely predictable logic, or work that cannot tolerate extra latency.
That is why an AI function should be a step in the flow, not the whole flow. Deterministic code handles the predictable work. AI takes the steps that used to require custom logic or a person. The two interoperate because they share the same function interface.
Invoice processing is a good example of how Poly customers are using AI functions in production today. Retrieving emails, sending notifications, and routing approvals are predictable, and they should stay as traditional functions. Interpreting the invoice PDF itself is different. Formats vary, fields sit in different places, and the meaning of a line item often depends on context. An AI function can read the document, extract the required information, and return it in a structured format for the rest of the workflow.
When extraction criteria, routing rules, or approval requirements change, that logic lives in the prompt rather than in custom code a developer has to rewrite. The surrounding workflow stays the same: documents in, structured data out, approvals routed. AI is used only where interpretation is needed, and the broader process remains controlled, observable, and deterministic.
What’s next
To start, AI functions help most with highly variable use cases: the steps that are difficult to specify in advance, expensive to code by hand, and slow to change once they are written. That is how they improve time to market. You can ship the inference-heavy parts of a workflow quickly, watch how they behave in production, and learn where the real variance is.
Given the increased adoption of AI, there is already a lot of focus in the market about token optimization. Most of it, though, is about which model to use for which task and how to arm AI with context sufficient to improve performance. While this is important, we believe the more impactful optimization is to convert AI actions into deterministic code.
That is what is on the Poly roadmap. Poly is enhancing the AI function primitive to produce logs that another agent can consume and convert into standard, deterministic functions. Once a pattern is well understood, code will be more performant, inexpensive, and predictable than a model. AI remains valuable for getting to market and for understanding variance. In production, the goal is to compress its scope as much as possible so that the model only handles the edge cases that still require inference.
Looking for ways to optimize your runtime workflows, or already using AI agents? We’d love to talk about how Poly could help give you optionality and leverage over your tech stack. Reach out at hello@polyapi.io.