By now, you’re likely aware that the coming years will be defined by smart automations, from AI workflows and AI agents to agentic AI. These intelligent automations will integrate seamlessly into both programmatic processes and human-in-the-loop workflows.

To achieve high-quality outcomes and deploy these solutions successfully, investing in your data operations will be critical. Here are some key considerations from a data operations perspective:

Orchestration

• Implement a robust orchestration framework that can activate and track the results and errors across all tasks in your operational workflow.
• Ensure your orchestrator is agnostic to whether tasks are executed by machines or humans, adopting a black-box approach. Build a standardized interface for task inputs and outputs to enable seamless transitions between human and machine execution, as well as comparisons between them.

Data Quality

• Invest heavily in data organization, classification, cataloging, and indexing. The better your data quality, the more accurate and relevant your outputs will be, whether you’re using retrieval-augmented generation (RAG) techniques or fine-tuned models.

Quality Assurance (QA) and Quality Control (QC)

• Embed QA/QC processes into both your operational workflows and your final product.
• In the early stages, human oversight will likely be essential. Over time, as confidence grows in your workflows, you can gradually reduce human involvement.

Feedback Loops

• Use product interaction and QA/QC data as feedback to refine your models, improve controls, and optimize your operational flow. A strong feedback loop is key to continuous improvement.

Crawl-Walk-Run Approach

• Build confidence incrementally. Break your operational flow into manageable units that are easy to validate and compare across model versions.
• Prioritize obtaining real customer feedback as early as possible. This feedback, combined with robust QA/QC, will help identify which patterns work best and guide your focus for future investments.