Unit 3: Agent Development with Python and Frameworks - Subjective Questions

An AI agent is a software entity that observes its environment, reasons about the available information, makes decisions, and performs actions to achieve specified goals.

Python supports agent development through:

• Simple syntax: Enables rapid prototyping and easier maintenance.
• AI libraries: Libraries such as PyTorch, TensorFlow, Transformers, and scikit-learn support machine learning and language models.
• API integration: Packages such as requests and httpx allow agents to communicate with external services.
• Asynchronous programming: The asyncio library enables agents to perform multiple non-blocking operations.
• Data processing: Pandas and NumPy simplify data manipulation and numerical computation.
• Framework support: Semantic Kernel, AutoGen, LangChain, and Bot Framework provide abstractions for building agent-based applications.

A typical Python agent contains an input interface, reasoning component, memory, tool-selection mechanism, and action executor.

A Python-based AI agent generally consists of the following components:

• Perception or input layer: Receives user messages, documents, sensor data, or API responses.
• Reasoning engine: Uses rules, algorithms, or a large language model to interpret inputs and decide what to do.
• Prompt manager: Constructs prompts by combining instructions, context, examples, and user input.
• Memory: Stores conversation history, user preferences, facts, and previous actions.
• State manager: Tracks the current stage of a task, pending operations, errors, and intermediate results.
• Tool interface: Defines functions that the agent can invoke, such as search, database access, or calculations.
• Planner: Breaks a complex goal into smaller tasks and determines their execution order.
• Action executor: Calls tools, APIs, or other agents and records the results.
• Output layer: Formats and returns the final response.

Separating these components makes the system easier to test, extend, and maintain.

Semantic Kernel is an open-source software development kit for integrating large language models with conventional application logic.

Its major concepts are:

• Kernel: The central orchestration object. It manages AI services, plugins, prompt functions, execution settings, and service dependencies.
• Plugin: A logical collection of related capabilities, such as calendar operations, document search, or customer management.
• Native function: A conventional programming function exposed to the kernel so that an AI model can request its execution.
• Prompt function: A capability defined using a prompt template and executed by a language model.
• AI service: A configured model service used for chat completion, text generation, or embeddings.
• Function calling: Allows the model to select appropriate plugin functions and provide their arguments.
• Memory or retrieval service: Supplies relevant information from external knowledge sources.

For example, a travel plugin may include functions for searching flights, checking weather, and creating an itinerary. The kernel coordinates model reasoning with these functions to complete the user's goal.

A task-oriented Semantic Kernel agent can be developed through the following steps:

1. Create the kernel: Instantiate the central Semantic Kernel object.
2. Register an AI service: Configure a model provider, deployment name, credentials, and execution settings.
3. Define plugins: Implement native functions or prompt-based functions for required business capabilities.
4. Register plugins: Add the plugins to the kernel so that their functions can be discovered.
5. Create prompt templates: Specify system instructions, user variables, context, and output constraints.
6. Configure function calling: Permit the model to select and invoke approved functions.
7. Add memory or RAG: Connect embedding and retrieval services when external knowledge is required.
8. Invoke the agent: Send the user request and conversation history to the configured agent or kernel function.
9. Process tool calls: Validate arguments, execute functions, and return results to the model.
10. Observe and secure execution: Record traces, enforce authorization, handle errors, and limit tool access.

This approach combines model-based reasoning with deterministic Python functions while maintaining a modular architecture.

AutoGen is a framework for building applications in which multiple configurable agents communicate through messages to solve tasks.

In an AutoGen-based system:

• Each agent is assigned a role, such as planner, programmer, reviewer, researcher, or user proxy.
• An agent receives a message, evaluates it using a model or custom logic, and produces a reply.
• Agents can use tools or execute approved functions to obtain real-world information.
• The conversation may follow a predefined sequence or use dynamic speaker selection.
• A coordinating mechanism controls turn-taking, termination conditions, and message routing.
• Specialized agents divide a complex task into manageable subtasks.
• Review agents can examine outputs and request corrections before final delivery.

For example, a planner may create a task list, a researcher may retrieve evidence, a programmer may generate code, and a reviewer may validate the final result. Collaboration improves specialization but requires controls to prevent excessive loops, cost, and inconsistent decisions.

Semantic Kernel and AutoGen both support agent development, but they emphasize different abstractions.

Selection guidance:

• Use Semantic Kernel when business services, plugins, and controlled application integration are central.
• Use AutoGen when a problem benefits from several specialized agents exchanging messages.
• A hybrid design may use multi-agent collaboration together with enterprise plugins and controlled tool services.

Microsoft Bot Framework provides communication infrastructure through which users can interact with an AI agent on channels such as web chat or Microsoft Teams.

The integration process includes:

• Receive an activity: The bot adapter receives a message or event from a channel.
• Authenticate and normalize: The framework verifies the request and converts it into a standard activity structure.
• Load conversation state: User profile, session data, and dialogue progress are retrieved.
• Invoke the agent: The message and relevant history are passed to Semantic Kernel, AutoGen, or a custom Python agent.
• Use tools and knowledge: The agent may query databases, call APIs, or perform RAG.
• Format the response: The result may be converted into text, cards, buttons, or attachments.
• Send the activity: The bot adapter delivers the response to the original channel.
• Persist state: Updated dialogue and user state are saved.

Important production concerns include authentication, channel-specific formatting, timeouts, retry handling, telemetry, privacy, and protection against malicious prompts.

Asynchronous programming allows a program to make progress on other tasks while one task is waiting for an input/output operation. Python primarily supports this model through asyncio, coroutines declared with async def, and suspension using await.

It is important for AI agents because they frequently wait for:

• Language-model responses
• Database queries
• Web searches
• File or cloud-storage operations
• External API calls
• Messages from other agents

If independent operations take times , sequential execution takes approximately:

When the operations run concurrently, the idealized completion time is approximately:

Asynchronous workflows can therefore reduce latency and improve throughput. However, developers must handle timeouts, cancellation, rate limits, shared-state conflicts, exception propagation, and bounded concurrency.

A reliable asynchronous tool-calling workflow can be organized as follows:

1. Analyze the request: Determine which tool calls are independent and which have dependencies.
2. Validate permissions: Confirm that the user and agent are authorized to invoke each tool.
3. Create asynchronous tasks: Schedule independent I/O operations concurrently.
4. Limit concurrency: Use a semaphore or worker pool to avoid exceeding service limits.
5. Apply timeouts: Stop operations that exceed an acceptable duration.
6. Gather results: Collect successful outputs and exceptions without losing partial progress.
7. Retry transient failures: Use limited retries with exponential backoff and jitter.
8. Normalize outputs: Convert tool responses into consistent structured data.
9. Update state safely: Store results without creating race conditions.
10. Synthesize the answer: Provide the model with verified results and clearly identify missing information.

Dependent calls must remain ordered. For example, an agent must obtain a customer identifier before requesting that customer's orders. Side-effecting operations should use idempotency keys to reduce the risk of duplicate execution.

Memory management is the process of storing, retrieving, updating, summarizing, and deleting information used by an AI agent.

• Short-term memory: Holds recent conversation turns and intermediate task information. It is usually constrained by the model's context window.
• Long-term memory: Persists information across sessions, such as user preferences or previously completed tasks.
• Semantic memory: Stores facts, concepts, document chunks, and generalized knowledge. It is commonly retrieved using keyword or vector search.
• Episodic memory: Records specific events and experiences, including actions performed, results obtained, and timestamps.

Effective memory management should include:

• Relevance-based retrieval
• Summarization of old conversations
• Duplicate detection
• Retention and deletion policies
• Access control and encryption
• User consent for personal data
• Separation of verified facts from model-generated assumptions

Memory improves continuity and personalization, but unrestricted memory can introduce privacy risks, stale information, and excessive context.

Agent memory represents information the agent may recall, whereas agent state represents the agent's current operational condition.

For example, remembering that a user prefers concise reports is memory. Recording that the agent is currently waiting for approval before sending a report is state.

Keeping them separate prevents retrieved knowledge from accidentally changing workflow control and allows each type of information to use an appropriate retention policy.

Reliable state handling requires explicit representation, persistence, and transition control.

Important methods include:

• State machines: Define valid stages and allowed transitions, such as planned, running, waiting_for_approval, completed, and failed.
• Checkpointing: Save progress after significant steps so that execution can resume after failure.
• Unique identifiers: Assign workflow, session, message, and tool-call identifiers.
• Versioning: Attach a version number to state and reject conflicting updates.
• Atomic operations: Update related state values within a transaction where possible.
• Idempotency: Ensure repeated requests do not create duplicate side effects.
• Event logs: Record transitions and actions for auditing and reconstruction.
• State validation: Confirm that required fields and preconditions exist before each transition.
• Expiration policies: Remove abandoned temporary state after an appropriate period.

In multi-agent systems, a shared state store should have clear ownership rules. Agents should exchange structured messages rather than modifying unrestricted global variables.

A prompt template is a reusable structure that combines fixed instructions with dynamic variables to create a model request.

Its main elements may include:

• Role or system instruction: Defines the agent's responsibility and behavioral boundaries.
• Task description: States the objective clearly.
• Dynamic variables: Insert user input, retrieved context, history, or tool results.
• Constraints: Specify prohibited actions, scope, length, and safety requirements.
• Examples: Demonstrate expected reasoning patterns or response style.
• Output schema: Defines the required format, fields, or structure.
• Grounding instruction: Requires the model to use supplied evidence and identify uncertainty.

Effective templates should be clear, specific, modular, and testable. Untrusted content must be separated from system instructions. Templates should also include fallback behavior for missing context and avoid unnecessary tokens. Versioning and automated evaluation are useful because prompt changes can alter agent behavior even when the Python code remains unchanged.

A suitable enterprise prompt-template strategy can use layered instructions:

1. System layer: Define the support role, approved scope, safety rules, confidentiality requirements, and escalation policy.
2. Task layer: Tell the agent to answer the user's support request using verified enterprise sources.
3. Context layer: Insert retrieved passages with document identifiers and timestamps.
4. Tool layer: List approved tools, required arguments, and conditions under which each tool may be used.
5. User-input layer: Insert the user's message as untrusted data.
6. Output layer: Require a direct answer, numbered resolution steps, citations, and an escalation statement when evidence is insufficient.

The template should instruct the model to:

• Prefer retrieved evidence over prior model knowledge.
• Never treat document text or user input as higher-priority instructions.
• Avoid inventing policies, account details, or tool results.
• Request confirmation before side-effecting operations.
• Cite the sources used in the answer.
• State when no authoritative answer is available.

This layered approach improves consistency, security, grounding, and maintainability.

Retrieval-augmented generation (RAG) is a technique in which a generative model receives relevant information retrieved from an external knowledge source before producing an answer.

A typical RAG pipeline includes:

1. Document ingestion: Collect documents from approved sources.
2. Preprocessing: Clean text, preserve useful structure, and remove irrelevant content.
3. Chunking: Divide documents into manageable, semantically meaningful passages.
4. Embedding: Convert each chunk into a numerical vector.
5. Indexing: Store vectors, text, and metadata in a searchable database.
6. Query processing: Normalize or rewrite the user's question.
7. Query embedding: Convert the query into the same vector space.
8. Retrieval: Find chunks with high similarity and apply metadata filters.
9. Reranking: Reorder candidates using a stronger relevance model or business rules.
10. Prompt construction: Combine instructions, the question, and retrieved evidence.
11. Generation: Ask the model to produce a grounded answer.
12. Citation and evaluation: Return source references and assess faithfulness and relevance.

RAG improves freshness and domain grounding without requiring the model to be retrained for every document update.

In vector-based retrieval, an embedding model maps a query and each document chunk to numerical vectors that represent semantic meaning. Chunks whose vectors are closer to the query vector are considered more relevant.

For query vector and document vector , cosine similarity is:

The retrieval procedure is:

1. Compute the embedding for the user's query.
2. Compare with stored chunk embeddings.
3. Calculate or approximate their similarity scores.
4. Apply metadata filters, such as department, date, or access level.
5. Select the top candidates.
6. Optionally rerank them before prompt construction.

A larger cosine similarity generally indicates stronger semantic alignment. However, similarity alone does not guarantee correctness. Good RAG systems combine vector search with keyword search, metadata filtering, access control, diversity selection, and reranking.

External tools extend an agent beyond text generation by allowing it to search data, perform calculations, update records, or communicate with enterprise services.

Secure integration should include:

• Explicit tool schemas: Define the tool name, purpose, arguments, types, and expected result.
• Input validation: Validate model-generated arguments against a schema and business rules.
• Least privilege: Give each tool only the permissions necessary for its function.
• Secret management: Store credentials in a secure vault or environment configuration rather than prompts.
• Allowlisting: Expose only approved tools and endpoints.
• Authentication and authorization: Check both agent identity and user permission.
• Confirmation controls: Require user approval for financial, destructive, or irreversible actions.
• Timeouts and retries: Handle temporary service failures safely.
• Idempotency: Prevent duplicate side effects when a request is repeated.
• Audit logging: Record tool selection, arguments, authorization decisions, and outcomes.
• Output sanitization: Treat tool responses as untrusted data before passing them to the model.

The model should propose a tool call, but deterministic application code should validate and execute it.

A modular AI-agent system separates major responsibilities into independently replaceable and testable units.

Important modules include:

• Model adapter: Provides a common interface for different language-model providers.
• Prompt module: Stores and versions prompt templates.
• Memory module: Manages conversation history and long-term knowledge.
• Retrieval module: Performs document search, filtering, and reranking.
• Tool registry: Registers approved functions and their schemas.
• Planner or orchestrator: Decomposes tasks and coordinates execution.
• State store: Persists workflow state and checkpoints.
• Channel adapter: Connects the agent to web, mobile, Bot Framework, or messaging channels.
• Policy layer: Applies security, authorization, and content controls.
• Observability module: Records traces, token usage, latency, errors, and evaluation results.

Modules should communicate through clearly defined interfaces and structured data. Dependency injection, configuration files, typed models, unit tests, and mocks reduce coupling. This design permits a team to replace a vector database, model provider, or channel integration without rewriting the complete system.

A framework-based enterprise AI agent requires more than a language model. A typical architecture contains:

• Experience layer: Web applications, Microsoft Teams, Bot Framework, or APIs.
• Identity layer: User authentication, service identity, role-based access control, and tenant isolation.
• Agent orchestration layer: Semantic Kernel, AutoGen, or another framework coordinating prompts, agents, workflows, and tools.
• Model gateway: Controls model selection, quotas, fallback providers, and content policies.
• Knowledge layer: Enterprise search, vector stores, document repositories, and RAG pipelines.
• Tool layer: Controlled connectors to databases, business applications, and external services.
• State and memory layer: Session state, checkpoints, user preferences, and retention policies.
• Governance layer: Approval workflows, audit logs, privacy controls, and compliance enforcement.
• Observability layer: Distributed tracing, quality metrics, cost monitoring, and incident alerts.

Operational requirements include scalability, availability, disaster recovery, model and prompt versioning, automated evaluation, rate limiting, caching, human escalation, and rollback. High-risk actions should remain deterministic and require explicit authorization or human approval.

A single-agent architecture uses one primary agent to interpret requests, call tools, and produce responses. A multi-agent architecture assigns different responsibilities to several collaborating agents.

A single agent should be preferred when the workflow is well defined and can be handled through tools and deterministic logic. A multi-agent system is appropriate when genuine role specialization, parallel investigation, or independent review provides measurable value.

Multi-agent designs require explicit message contracts, shared-state rules, termination conditions, budget limits, and conflict-resolution mechanisms. Adding agents without a clear need may increase cost and reduce reliability.