Unit 3: Agent Development with Python and Frameworks

I. Orientation — Foundations of Agent Development

An AI agent is a software system that uses an AI model to interpret context, select actions, invoke tools, maintain relevant information, and work toward a goal. Modern agent development combines probabilistic model reasoning with deterministic software components such as APIs, databases, validation rules, and workflow engines.

• Core operating cycle: An agent commonly follows observe → reason → act → evaluate, repeating until it reaches a stopping condition.
• Model-driven control: A large language model can classify intent, generate plans, select tools, and formulate responses, but application code must constrain its permissions.
• Agent environment: Inputs may include user messages, retrieved documents, API results, system events, and stored state.
• Tools and actions: Functions, plugins, web services, databases, and business systems allow the agent to affect systems beyond text generation.
• Non-determinism: Identical inputs may produce different outputs; schemas, tests, validation, and bounded workflows improve reliability.
• Enterprise requirements: Production agents need authentication, authorization, observability, privacy controls, audit trails, fault handling, and human escalation.
• Framework role: Semantic Kernel, AutoGen, and Bot Framework provide reusable abstractions for orchestration, communication, tool invocation, and integration.

II. Python Foundations — Implementing the Agent Loop

A. Python programming for AI agents

Python supports agent development through concise syntax, asynchronous I/O, type annotations, extensive AI libraries, and straightforward API integration.

• Agent components: A practical design separates the model client, tool registry, memory provider, policy layer, and orchestration loop into independently testable units.
• Typed contracts: dataclasses, TypedDict, or Pydantic models define structured requests and responses. A tool argument such as order_id: str should be validated before execution.
• Tool registry: Functions can be stored by name so that a model-generated tool request maps to controlled application code.
• Dependency management: Virtual environments and pinned packages make builds reproducible; secrets belong in environment variables or managed secret stores.
• Error boundaries: Model failures, invalid JSON, API timeouts, and tool exceptions should be handled separately because each requires a different retry or fallback policy.
• Minimal agent loop:

TOOLS = {"lookup_order": lookup_order}

for step in range(MAX_STEPS):
    decision = await model.decide(messages, tools=TOOLS)
    if decision.final_answer:
        return decision.final_answer
    result = await TOOLS[decision.tool_name](**decision.arguments)
    messages.append({"role": "tool", "content": str(result)})

• Bounded execution: MAX_STEPS prevents an agent from continuing indefinitely when a task cannot be completed.

B. Asynchronous workflows

Asynchronous workflows let an agent wait for models, databases, and remote tools without blocking the Python process.

• Coroutine model: Functions declared with async def return coroutines; await suspends one coroutine while the event loop runs other work.
• Concurrent operations: Independent retrieval requests can run together with asyncio.gather, reducing total latency from approximately the sum of durations to the longest duration.
• Example:

profile, orders = await asyncio.gather(
    get_profile(user_id),
    get_orders(user_id),
)

• Control mechanisms: asyncio.timeout() bounds waiting time, while semaphores restrict concurrency and protect rate-limited services.
• Cancellation: Cancelled work should release database connections and network sessions through context managers or finally blocks.
• Ordering constraint: Dependent actions remain sequential; payment confirmation must complete before an order is marked as paid.
• Failure policy: Retries should use limited attempts and exponential backoff, especially for transient HTTP 429 and 5xx responses.

III. Semantic Kernel — Plugin-Based AI Orchestration

A. Semantic Kernel fundamentals

Semantic Kernel is an SDK for integrating AI models with application functions, prompts, plugins, and orchestration logic.

• Kernel: The kernel acts as a composition container through which configured AI services and registered plugins become available to an application.
• AI service: A chat-completion or embedding service connects the kernel to a chosen model provider and deployment.
• Plugins: A plugin groups related callable functions, such as InventoryPlugin.check_stock and InventoryPlugin.reserve_item.
• Functions: Kernel functions may wrap native code or prompt-based behavior; descriptions and parameter metadata help the model choose them.
• Function calling: The model can request a registered function, after which the application or framework executes it and returns the result to the conversation.
• Filters and controls: Invocation filters can implement logging, authorization, argument validation, and exception handling around function calls.
• Practical boundary: Semantic Kernel organizes model-service integration, but domain rules should remain in deterministic business services rather than prompts.

IV. AutoGen — Conversational Multi-Agent Coordination

A. AutoGen framework

AutoGen supports applications in which model-backed or code-backed agents exchange messages to complete tasks.

• Agent abstraction: Each agent has a role, instructions, capabilities, and message-handling behavior; examples include planner, researcher, executor, and reviewer.
• Conversation pattern: A task is represented as a sequence of messages rather than a single prompt, allowing agents to refine or evaluate intermediate outputs.
• Tool execution: An execution agent can run approved functions or code while another agent concentrates on reasoning or review.
• Coordination: Multi-agent workflows require rules for speaker selection, handoffs, termination, and maximum turns.
• Role separation: A reviewer agent may identify unsupported claims, but it does not guarantee correctness unless claims are checked against trusted evidence.
• Operational risk: More agents increase token consumption, latency, and failure paths; a single agent with tools is preferable when role separation adds no measurable value.
• Safety boundary: Code execution should occur in a restricted environment with resource limits, isolated files, and explicit network permissions.

V. Bot Framework — User-Channel Integration

A. Bot Framework integration

Bot Framework integration connects an agent to conversational channels while managing message delivery and interaction events.

• Activity model: Incoming activities can represent messages, conversation updates, attachments, or button actions, giving the bot a consistent event structure.
• Adapter layer: The adapter receives channel traffic, authenticates requests, and passes activities to bot logic.
• Turn context: A turn context exposes the current activity and methods for sending replies during that interaction.
• Conversation identity: Channel, conversation, and user identifiers help scope state correctly; they should not be treated as proof of business identity without authentication.
• Agent connection: Bot handlers can forward normalized user text to a Semantic Kernel or AutoGen workflow and translate the result into channel-compatible output.
• Channel constraints: Rich cards, attachments, message sizes, and interaction features vary across channels, so responses require capability-aware formatting.
• Enterprise controls: Production integration needs identity mapping, transcript policies, telemetry correlation IDs, rate limits, and escalation to human support.

VI. Context and Continuity — Managing Agent Information

A. Memory management

Memory management controls which past information is available to the agent and how it is stored, retrieved, and expired.

• Working memory: Recent messages and current tool results occupy the active context window and directly influence the next model response.
• Short-term memory: Conversation summaries compress older turns when full transcripts would exceed token limits.
• Long-term memory: Durable stores retain approved preferences, facts, or task history across sessions.
• Semantic memory: Embeddings represent content as vectors; similarity search retrieves items related in meaning to a query.
• Selection policy: Useful context should be relevant, recent, trustworthy, and permitted for the current user; storing everything increases cost and privacy risk.
• Lifecycle: Memory records need source metadata, ownership, timestamps, retention periods, and deletion mechanisms.
• Security rule: Retrieved memories are untrusted inputs and cannot override system instructions or authorization checks.

B. State handling

State handling records deterministic workflow data required to continue an interaction or recover after interruption.

• State categories: User state stores user-scoped settings, conversation state stores session data, and workflow state stores task progress.
• Memory distinction: State may contain approval_status = "pending"; memory may contain a summary explaining why approval was requested.
• Persistence: External storage such as Redis, SQL, or a managed state service supports recovery across processes and deployments.
• Concurrency control: Version numbers, optimistic locking, or transactions prevent simultaneous turns from overwriting one another.
• State machine: Explicit states such as COLLECTING_DETAILS, AWAITING_APPROVAL, and COMPLETED constrain valid transitions.
• Idempotency: A unique operation key ensures that retrying create_invoice does not create duplicate invoices.
• Minimization: Sensitive values should be encrypted or omitted, and expired workflow state should be removed.

VII. Grounded Reasoning — Prompts, Retrieval, and Tools

A. Prompt templates

Prompt templates provide parameterized instructions that make model interactions consistent and maintainable.

• Template structure: A template typically defines role, objective, constraints, context, input variables, and required output format.
• Variables: Placeholders such as {{$customer_query}} separate reusable instructions from runtime data.
• Instruction hierarchy: System-level policy should remain distinct from user content and retrieved documents.
• Structured output: Requiring a JSON schema such as {"category": str, "confidence": float} makes responses easier to validate and route.
• Delimiters: Clearly marked context blocks reduce ambiguity, although delimiters alone do not prevent prompt injection.
• Versioning: Prompts should be stored, reviewed, tested, and associated with model settings such as temperature and token limits.
• Evaluation: Test sets should measure task success, groundedness, formatting compliance, unsafe output, latency, and cost.

B. Retrieval-augmented generation (RAG)

RAG supplies a model with selected external evidence at request time so that responses can use current, domain-specific information.

• Ingestion pipeline: Documents are parsed, divided into chunks, enriched with metadata, embedded, and indexed.
• Retrieval pipeline: The query is embedded, candidate chunks are found through vector or hybrid search, and results may be reranked.
• Generation pipeline: Retrieved passages are inserted into a prompt that directs the model to answer from the supplied evidence.
• Similarity measure: Cosine similarity compares query vector (q) and document vector (d):

similarity(q, d) = (q · d) / (||q|| ||d||)

Here, q · d is the dot product and ||q||, ||d|| are vector magnitudes.

• Grounding: Source identifiers and quoted evidence support citations and allow users or evaluators to verify an answer.
• Limitations: Poor chunking, stale indexes, missing documents, or irrelevant retrieval can produce unsupported answers despite a strong model.
• Access control: Retrieval filters must enforce document permissions before content enters the model context.

C. Integration of external tools

External tools allow agents to retrieve authoritative data and perform controlled actions through APIs, functions, and enterprise services.

• Tool contract: Each tool needs a clear name, description, typed parameters, return schema, and documented failure behavior.
• Validation: Application code must reject malformed arguments and values outside allowed ranges before invoking the service.
• Authorization: The agent may propose an action, but the execution layer must verify the user’s permission for the specific resource.
• Read-write distinction: Search and lookup tools are lower risk than payment, deletion, or messaging tools.
• Approval gates: Irreversible or high-impact actions should require explicit confirmation that includes the exact target and effect.
• Resilience: Timeouts, circuit breakers, idempotency keys, and bounded retries protect dependencies and prevent duplicate side effects.
• Observability: Logs should capture tool name, duration, outcome, and correlation ID while excluding secrets and unnecessary personal data.

VIII. System Architecture — Composable Enterprise Agents

A. Building modular AI systems

Modular AI systems isolate responsibilities so that models, tools, data stores, and policies can evolve independently.

• Module boundaries: Typical modules include channel adapters, orchestrators, model gateways, retrievers, memory stores, tool services, and policy enforcement.
• Interfaces: Stable typed contracts allow a retriever or model provider to be replaced without rewriting the workflow.
• Dependency inversion: Agent logic should depend on abstractions such as Retriever.search(query) rather than a particular vector database SDK.
• Deterministic core: Pricing, permissions, workflow transitions, and validation remain in code; models handle language interpretation and flexible reasoning.
• Testing layers: Unit tests cover pure functions, contract tests cover integrations, and scenario evaluations assess end-to-end agent behavior.
• Fallback design: Modules can return cached data, request clarification, transfer to a human, or fail closed when a dependency is unavailable.
• Deployment benefit: Independent components support controlled releases, scaling, monitoring, and ownership by separate teams.

B. Framework-based enterprise agent development

Framework-based enterprise development applies reusable orchestration components while preserving organizational controls and operational reliability.

• Framework selection: Semantic Kernel suits plugin-oriented model integration, AutoGen emphasizes agent conversations, and Bot Framework provides conversational channel infrastructure.
• Layered architecture: Channels call an agent service; the service invokes approved models, retrieval systems, tools, and state stores through governed interfaces.
• Identity and access: Single sign-on, delegated credentials, role-based access control, and tenant isolation must apply to every retrieval and action.
• Governance: Approved model versions, prompt registries, data classifications, retention rules, and human-approval policies create auditable boundaries.
• Observability: Distributed traces should connect a user turn to model calls, retrieval results, tool invocations, token usage, latency, and failures.
• Evaluation pipeline: Automated regression suites compare groundedness, task completion, safety, latency, and cost before deployment.
• Operational strategy: Versioned prompts, feature flags, canary releases, rollback procedures, and human escalation reduce production risk.
• Design principle: Frameworks accelerate implementation, but enterprise correctness ultimately depends on explicit policies, deterministic controls, secure integrations, and continuous evaluation.