OpenAI Integration Guide

Comprehensive guide to OpenAI's API and Assistants and how they integrate with ayaiay packs

Table of Contents

• Overview
• Core Concepts
  • Assistants
  • Threads
  • Functions
  • Retrieval
  • Code Interpreter
  • Vision
• ayaiay Integration
• Best Practices
• Examples
• Official References

---

Overview

OpenAI provides powerful AI models and APIs for building AI applications. With GPT-4, GPT-4 Turbo, and the Assistants API, OpenAI enables developers to create sophisticated AI agents with persistent state, tool use, and custom knowledge.

What is OpenAI?

OpenAI offers:

• Powerful Models: GPT-4, GPT-4 Turbo, GPT-3.5 Turbo
• Assistants API: Persistent AI agents with state management
• Function Calling: Structured tool invocation
• Multimodal: Text, image, and voice capabilities
• Retrieval: Built-in document search and knowledge bases
• Production-Ready: Battle-tested at scale

Model Family

Model	Context Window	Best For	Cost
GPT-4 Turbo	128K tokens	Complex tasks, long documents	$$
GPT-4	8K/32K tokens	High-quality responses	$$$
GPT-3.5 Turbo	16K tokens	Fast, cost-effective tasks	$
GPT-4 Vision	128K tokens	Image understanding	$$
GPT-4o	128K tokens	Multimodal (text, vision, audio)	$$

Key Capabilities

• Assistants: Persistent agents with custom instructions and tools
• Threads: Conversation history management
• Functions: Custom tool integration with JSON Schema
• Retrieval: Automatic document search across uploaded files
• Code Interpreter: Execute Python code safely
• Vision: Image understanding and analysis
• DALL-E: Image generation
• Whisper: Speech-to-text transcription
• TTS: Text-to-speech synthesis

---

Core Concepts

Assistants

Assistants are persistent AI agents configured with custom instructions, tools, and knowledge.

What are Assistants?

Assistants:

• Have persistent configuration and state
• Can use multiple tools simultaneously
• Maintain conversation history in Threads
• Access uploaded files for retrieval
• Execute Python code with Code Interpreter

Creating an Assistant

from openai import OpenAI

client = OpenAI()

assistant = client.beta.assistants.create(
    name="Python Code Reviewer",
    instructions="""
    You are an expert Python developer specializing in code review.

    ## Your Role
    - Review Python code for best practices
    - Identify bugs and security issues
    - Suggest performance improvements
    - Follow PEP 8 style guidelines

    ## Response Format
    - Be concise and actionable
    - Provide code examples
    - Explain reasoning
    - Prioritize critical issues
    """,
    model="gpt-4-turbo",
    tools=[
        {"type": "code_interpreter"},
        {"type": "retrieval"}
    ]
)

Assistant Configuration

assistant = client.beta.assistants.create(
    name="Data Analyst",
    instructions="You are a data analysis expert...",
    model="gpt-4-turbo",
    tools=[
        {"type": "code_interpreter"},  # Execute Python
        {"type": "retrieval"},         # Search files
        {
            "type": "function",        # Custom function
            "function": {
                "name": "query_database",
                "description": "Query the SQL database",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {"type": "string"}
                    },
                    "required": ["query"]
                }
            }
        }
    ],
    file_ids=[file1.id, file2.id],  # Knowledge base files
    metadata={"project": "data-analysis", "version": "1.0"}
)

---

Threads

Threads manage conversation history and context between users and assistants.

What are Threads?

Threads:

• Store message history
• Maintain conversation context
• Support multiple messages
• Can be resumed across sessions
• Automatically manage token limits

Using Threads

# Create a thread
thread = client.beta.threads.create()

# Add messages
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Review this Python code: ..."
)

# Run the assistant
run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id
)

# Wait for completion
while run.status != "completed":
    run = client.beta.threads.runs.retrieve(
        thread_id=thread.id,
        run_id=run.id
    )
    time.sleep(1)

# Get response
messages = client.beta.threads.messages.list(
    thread_id=thread.id
)

print(messages.data[0].content[0].text.value)

Thread Management

# Resume existing thread
existing_thread = client.beta.threads.retrieve(thread_id)

# Add new message
client.beta.threads.messages.create(
    thread_id=existing_thread.id,
    role="user",
    content="Now review the updated code"
)

# Delete thread when done
client.beta.threads.delete(thread_id)

---

Functions

Functions enable assistants to call custom tools and APIs with structured inputs.

How Functions Work

1. Define functions with JSON Schema
2. Assistant analyzes user input
3. Assistant generates function call with arguments
4. Your code executes the function
5. Return results to assistant
6. Assistant incorporates results into response

Defining Functions

functions = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City name or coordinates"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_database",
            "description": "Search product database",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "Search query"
                    },
                    "category": {
                        "type": "string",
                        "description": "Product category filter"
                    }
                },
                "required": ["query"]
            }
        }
    }
]

assistant = client.beta.assistants.create(
    name="Shopping Assistant",
    instructions="Help users find products...",
    model="gpt-4-turbo",
    tools=functions
)

Handling Function Calls

# Create run
run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id
)

# Check for function calls
while run.status != "completed":
    run = client.beta.threads.runs.retrieve(
        thread_id=thread.id,
        run_id=run.id
    )

    if run.status == "requires_action":
        tool_calls = run.required_action.submit_tool_outputs.tool_calls
        tool_outputs = []

        for tool_call in tool_calls:
            function_name = tool_call.function.name
            function_args = json.loads(tool_call.function.arguments)

            # Execute function
            if function_name == "get_weather":
                result = get_weather(**function_args)
            elif function_name == "search_database":
                result = search_database(**function_args)

            tool_outputs.append({
                "tool_call_id": tool_call.id,
                "output": json.dumps(result)
            })

        # Submit results
        run = client.beta.threads.runs.submit_tool_outputs(
            thread_id=thread.id,
            run_id=run.id,
            tool_outputs=tool_outputs
        )

    time.sleep(1)

---

Retrieval

Retrieval enables assistants to search and access content from uploaded files.

What is Retrieval?

Retrieval allows assistants to:

• Search across multiple documents
• Find relevant information automatically
• Cite sources in responses
• Handle large knowledge bases

Using Retrieval

# Upload files
file1 = client.files.create(
    file=open("documentation.pdf", "rb"),
    purpose="assistants"
)

file2 = client.files.create(
    file=open("guidelines.md", "rb"),
    purpose="assistants"
)

# Create assistant with retrieval
assistant = client.beta.assistants.create(
    name="Documentation Assistant",
    instructions="Answer questions based on the uploaded documentation",
    model="gpt-4-turbo",
    tools=[{"type": "retrieval"}],
    file_ids=[file1.id, file2.id]
)

# Assistant automatically searches files when needed
thread = client.beta.threads.create()
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="What are the authentication requirements?"
)

run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id
)
# Assistant searches files and provides answer with citations

Best Practices for Retrieval

• Chunk documents appropriately: Break large files into logical sections
• Use clear file names: Helps with source attribution
• Update files regularly: Keep knowledge base current
• Limit file count: Performance degrades with 100+ files
• Supported formats: PDF, TXT, MD, DOCX, HTML

---

Code Interpreter

Code Interpreter allows assistants to write and execute Python code safely.

What is Code Interpreter?

Code Interpreter enables assistants to:

• Execute Python code in a sandboxed environment
• Generate charts and visualizations
• Process data files (CSV, Excel, JSON)
• Perform calculations and analysis
• Create and download files

Using Code Interpreter

assistant = client.beta.assistants.create(
    name="Data Analyst",
    instructions="Analyze data and create visualizations",
    model="gpt-4-turbo",
    tools=[{"type": "code_interpreter"}]
)

# Upload data file
file = client.files.create(
    file=open("sales_data.csv", "rb"),
    purpose="assistants"
)

# Create thread with file
thread = client.beta.threads.create()
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Analyze this sales data and create a monthly revenue chart",
    file_ids=[file.id]
)

# Assistant will:
# 1. Read the CSV file
# 2. Process the data
# 3. Generate visualization
# 4. Provide insights

Example Output

# Assistant generates and runs code like:
import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('sales_data.csv')
monthly_revenue = df.groupby('month')['revenue'].sum()
monthly_revenue.plot(kind='bar')
plt.title('Monthly Revenue')
plt.savefig('revenue_chart.png')

# Chart is available for download
# Assistant provides insights about the data

---

Vision

Vision capabilities allow assistants to understand and analyze images.

What is Vision?

GPT-4 Vision can:

• Describe images in detail
• Read text from images (OCR)
• Identify objects and scenes
• Analyze diagrams and charts
• Compare multiple images

Using Vision

response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's in this image? Describe it in detail."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/image.jpg"
                    }
                }
            ]
        }
    ]
)

print(response.choices[0].message.content)

Multiple Images

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Compare these UI mockups and identify differences"},
            {"type": "image_url", "image_url": {"url": "mockup1.jpg"}},
            {"type": "image_url", "image_url": {"url": "mockup2.jpg"}}
        ]
    }
]

---

ayaiay Integration

Concept Mapping

ayaiay Concept	OpenAI Equivalent	Description
Pack	Assistant + Configuration	Complete AI setup
Agent	Assistant	Persistent AI with tools
Instructions	Assistant Instructions	Behavior configuration
Tools	Functions + Tools	Callable capabilities
Context	Thread Messages	Conversation history
Knowledge	Retrieval + Files	Document knowledge base

Pack to OpenAI Translation

ayaiay Pack Structure

# pack.yaml
name: code-reviewer
version: 1.0.0
description: Expert code reviewer
model: gpt-4-turbo

system_prompt: |
  You are an expert code reviewer...

tools:
  - name: analyze_complexity
    description: Calculate code complexity
  - name: run_tests
    description: Execute test suite

files:
  - docs/coding-standards.md
  - docs/best-practices.md

Equivalent OpenAI Implementation

from openai import OpenAI

client = OpenAI()

# Upload knowledge files
file_ids = []
for filepath in ["docs/coding-standards.md", "docs/best-practices.md"]:
    file = client.files.create(
        file=open(filepath, "rb"),
        purpose="assistants"
    )
    file_ids.append(file.id)

# Define custom functions
functions = [
    {
        "type": "function",
        "function": {
            "name": "analyze_complexity",
            "description": "Calculate code complexity",
            "parameters": {
                "type": "object",
                "properties": {
                    "code": {"type": "string"},
                    "language": {"type": "string"}
                },
                "required": ["code"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "run_tests",
            "description": "Execute test suite",
            "parameters": {
                "type": "object",
                "properties": {
                    "test_path": {"type": "string"}
                },
                "required": ["test_path"]
            }
        }
    }
]

# Create assistant
assistant = client.beta.assistants.create(
    name="Code Reviewer",
    instructions="You are an expert code reviewer...",
    model="gpt-4-turbo",
    tools=[
        {"type": "retrieval"},      # Access knowledge files
        {"type": "code_interpreter"} # Execute code
    ] + functions,
    file_ids=file_ids
)

# Use assistant
thread = client.beta.threads.create()
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Review this Python code: ..."
)

run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id
)

---

Best Practices

1. Use Assistants for Persistent Agents

# Create once, use many times
assistant = client.beta.assistants.create(
    name="Customer Support",
    instructions="...",
    model="gpt-4-turbo"
)

# Reuse across sessions
thread = client.beta.threads.create()
# ... use assistant with thread

2. Manage Threads Efficiently

# Store thread IDs per user/session
user_threads = {}

def get_or_create_thread(user_id):
    if user_id not in user_threads:
        thread = client.beta.threads.create()
        user_threads[user_id] = thread.id
    return user_threads[user_id]

3. Handle Function Calls Robustly

def execute_function(function_name, arguments):
    try:
        if function_name == "get_weather":
            return get_weather(**arguments)
        elif function_name == "search_db":
            return search_database(**arguments)
        else:
            return {"error": f"Unknown function: {function_name}"}
    except Exception as e:
        return {"error": str(e)}

4. Optimize Retrieval Performance

# Split large documents into smaller files
# Use descriptive file names
# Organize by topic

files = [
    "auth/authentication.md",
    "auth/authorization.md",
    "api/endpoints.md",
    "api/rate-limiting.md"
]

for filepath in files:
    file = client.files.create(
        file=open(filepath, "rb"),
        purpose="assistants"
    )
    file_ids.append(file.id)

5. Implement Proper Error Handling

def run_assistant(thread_id, assistant_id, max_retries=3):
    for attempt in range(max_retries):
        try:
            run = client.beta.threads.runs.create(
                thread_id=thread_id,
                assistant_id=assistant_id
            )

            while run.status not in ["completed", "failed", "cancelled"]:
                run = client.beta.threads.runs.retrieve(
                    thread_id=thread_id,
                    run_id=run.id
                )

                if run.status == "requires_action":
                    # Handle function calls
                    handle_required_action(run)

                time.sleep(1)

            if run.status == "completed":
                return get_messages(thread_id)
            else:
                raise Exception(f"Run failed with status: {run.status}")

        except Exception as e:
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)  # Exponential backoff
                continue
            raise

---

Examples

Example 1: Customer Support Assistant

assistant = client.beta.assistants.create(
    name="Customer Support",
    instructions="""
    You are a helpful customer support agent.

    Always be polite, professional, and empathetic.
    Search the knowledge base for answers.
    Escalate to human if unable to resolve.
    """,
    model="gpt-4-turbo",
    tools=[
        {"type": "retrieval"},
        {
            "type": "function",
            "function": {
                "name": "create_ticket",
                "description": "Create support ticket",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "title": {"type": "string"},
                        "description": {"type": "string"},
                        "priority": {"type": "string", "enum": ["low", "medium", "high"]}
                    },
                    "required": ["title", "description"]
                }
            }
        }
    ],
    file_ids=[faq_file.id, troubleshooting_file.id]
)

Example 2: Data Analysis Assistant

assistant = client.beta.assistants.create(
    name="Data Analyst",
    instructions="""
    You are a data analysis expert.

    Analyze data files and provide insights.
    Create visualizations when helpful.
    Explain your methodology clearly.
    """,
    model="gpt-4-turbo",
    tools=[{"type": "code_interpreter"}]
)

# Upload data
data_file = client.files.create(
    file=open("sales_data.csv", "rb"),
    purpose="assistants"
)

# Analyze
thread = client.beta.threads.create()
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Analyze sales trends and identify top performing products",
    file_ids=[data_file.id]
)

Example 3: Code Review Assistant

assistant = client.beta.assistants.create(
    name="Code Reviewer",
    instructions="""
    You are a senior software engineer.

    Review code for:
    - Best practices
    - Security vulnerabilities
    - Performance issues
    - Code style

    Provide specific, actionable feedback.
    """,
    model="gpt-4-turbo",
    tools=[
        {"type": "code_interpreter"},
        {
            "type": "function",
            "function": {
                "name": "run_linter",
                "description": "Run code linter",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "code": {"type": "string"},
                        "language": {"type": "string"}
                    },
                    "required": ["code", "language"]
                }
            }
        }
    ]
)

Example 4: Multimodal UI Review

response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Review this UI design and implementation. Identify discrepancies."
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "design_mockup.png"}
                },
                {
                    "type": "text",
                    "text": f"Implementation:\n```tsx\n{ui_code}\n```"
                }
            ]
        }
    ]
)

---

Official References

Documentation

• OpenAI Platform - Main dashboard
• API Reference - Complete API docs
• Assistants Guide - Assistants API
• Function Calling - Tool integration
• Vision Guide - Image understanding

Guides & Tutorials

• Quickstart - Get started
• Best Practices - Production tips
• Prompt Engineering - Writing effective prompts
• Fine-tuning - Custom models
• Safety Best Practices - Build safely

Resources

• Pricing - Cost calculator
• Usage Dashboard - Monitor usage
• Playground - Test prompts
• Community Forum - Get help
• Cookbook - Code examples

---

Next Steps

• Create Your First Pack - Build an ayaiay pack for OpenAI
• ayaiay Pack Specification - Complete manifest reference
• Other Providers - Compare with other AI platforms
• Example Packs - Learn from real examples