Comprehensive guide for building production-grade AI-integrated backends with multi-provider support, intelligent fallback mechanisms, region configuration, prompt management with variables/tools, and session-based billing. Use when implementing AI features in Django/Python backends or designing LLM-powered API architectures with external data integration.
<essential_principles>
1. Multi-Provider Design Philosophy
2. Separation of Concerns
3. Configuration Over Code
is_active flag to switch between configurations instantly4. Prompt Engineering Best Practices
{{variable_name}})5. Fallback & Resilience
6. Security & Authentication
<system_architecture>
User Request (Chat Message)
↓
API Endpoint (validate input, check session)
↓
Session Manager (create/resume session, lock pricing)
↓
External API Layer (if needed: fetch astrology, weather, etc.)
├─→ Cache Check (multi-layer: instance → distributed → database)
├─→ External API Call (if cache miss)
└─→ Data Transformation Pipeline (validate → normalize → enrich → format)
↓
Prompt Builder
├─→ Load Bot LLM Config (model, provider, temperature, metadata)
├─→ Render Prompt Template (replace {{variables}} with actual values)
├─→ Assemble Context (user data + external data + history)
└─→ Build Final Prompt (YAML or JSON structured)
↓
LLM Gateway Manager
├─→ Get/Generate JWT Token (cached)
├─→ Build Fallback Chain (provider-specific + universal)
└─→ POST to Internal Gateway (/v1/generate/)
↓
Internal LLM Gateway (separate service)
├─→ Validate JWT
├─→ Route to Primary Provider (OpenAI, Anthropic, Gemini, Bedrock, etc.)
├─→ On Failure: Try Fallback 1
├─→ On Failure: Try Fallback 2
└─→ On Failure: Universal Fallback (Gemini-Flash)
↓
Response Processing
├─→ Parse LLM Response
├─→ Validate Output
└─→ Store in Database
↓
Billing (if session-based)
├─→ Calculate Units (tokens, messages, minutes)
├─→ Check Wallet Balance
├─→ Deduct Amount
└─→ Log Transaction
↓
Save to Database (conversation history)
↓
Return to User (API response)
Request → Load BotLLMConfig → Determine Primary Provider
↓
Primary: Gemini 2.5-Pro
├─→ Fallback 1: Gemini 2.5-Flash
├─→ Fallback 2: Gemini 2.0-Flash
└─→ Universal Fallback: Gemini 2.0-Flash
Primary: AWS Bedrock Deepseek-v3 (region: ap-south-1)
├─→ Fallback 1: Byteplus Deepseek-v3
├─→ Fallback 2: Gemini 2.0-Flash (region-agnostic)
└─→ Universal Fallback: Gemini 2.0-Flash
Primary: Byteplus
├─→ Fallback 1: Gemini 2.0-Flash
└─→ Universal Fallback: Gemini 2.0-Flash
</system_architecture>
<domain_knowledge_index> All implementation details, code examples, and architectural patterns are organized below:
System Architecture:
Provider Integration:
Prompt Management:
External API Integration:
Production Considerations:
Code Examples:
<database_schema>
class Bot(models.Model):
uuid = models.UUIDField(default=uuid4, editable=False, unique=True)
user = models.ForeignKey("users.UserProfile", on_delete=models.CASCADE, related_name="bots")
# Identity
name = models.CharField(max_length=255)
slug = models.SlugField(max_length=255, unique=True, blank=True, db_index=True)
description = models.TextField(blank=True)
avatar = models.CharField(max_length=255, null=True, blank=True)
# Status and Type
status = models.CharField(max_length=20, choices=BotStatus.choices, default=BotStatus.DRAFT, db_index=True)
bot_type = models.CharField(max_length=30, choices=BotType.choices, default=BotType.COMPANION)
# Configuration
rank = models.PositiveIntegerField(default=0, db_index=True)
metadata = models.JSONField(default=dict, blank=True) # Store required_fields, pricing, etc.
# Timestamps
launched_on = models.DateTimeField(null=True, blank=True, db_index=True)
created_on = models.DateTimeField(auto_now_add=True)
updated_on = models.DateTimeField(auto_now=True)
deleted_at = models.DateTimeField(null=True, blank=True, db_index=True)
Key Design Decisions:
metadata JSONField for flexible per-bot configuration (required_fields, character_rating, pricing)deleted_at timestamp instead of hard deleteslug auto-generated from name for SEO-friendly URLsrank for custom ordering in UIclass BotLLMConfig(models.Model):
bot = models.ForeignKey(Bot, on_delete=models.CASCADE, related_name="llm_configs")
# LLM Configuration
prompt = models.TextField(blank=True) # Can be template with {{variables}}
model_name = models.CharField(max_length=100) # e.g., "gpt-4", "claude-3-sonnet", "gemini-2.5-pro"
llm_provider = models.CharField(max_length=50, choices=BotLLMProvider.choices) # openai, anthropic, gemini, bedrock, etc.
# Generation Parameters
temperature = models.FloatField(default=0.7)
top_p = models.FloatField(default=0.95)
top_k = models.IntegerField(null=True, blank=True)
max_output_tokens = models.PositiveIntegerField(default=1024)
# Activation
is_active = models.BooleanField(default=False) # Only one active config per bot
# Additional Configuration
metadata = models.JSONField(default=dict, blank=True) # Store region_name, tools, etc.
created_on = models.DateTimeField(auto_now_add=True)
updated_on = models.DateTimeField(auto_now=True)
class Meta:
db_table = "llm_configurations"
ordering = ["-is_active", "created_on"]
def save(self, *args, **kwargs):
# Ensure only one active config per bot
if self.is_active:
BotLLMConfig.objects.filter(bot=self.bot, is_active=True).exclude(pk=self.pk).update(is_active=False)
super().save(*args, **kwargs)
Key Design Decisions:
Bot can have multiple BotLLMConfig entries for A/B testingis_active flag controls which config is used (only one active per bot)metadata stores provider-specific settings (e.g., region_name for AWS Bedrock)class BotLLMProvider(models.TextChoices):
OpenAI = "openai", _("OpenAI")
GEMINI = "gemini", _("Gemini")
ANTHROPIC = "anthropic", _("Anthropic")
GROQ = "groq", _("Groq")
AI_SDK = "ai-sdk", _("AI-SDK")
BEDROCK = "bedrock", _("Bedrock")
BYTEPLUS = "byteplus", _("Byteplus")
OPENROUTER = "openrouter", _("OpenRouter")
Why This Design:
<fallback_mechanisms>
def get_fallback_routing(provider: str, model_name: str, system_instruction: str, llm_metadata: dict = None):
"""
Ensures system_instruction and region_name are added properly to fallback chain.
Args:
provider: Primary provider (gemini, bedrock, byteplus, etc.)
model_name: Model identifier (gemini-2.5-pro, deepseek.v3-v1:0, etc.)
system_instruction: System prompt to pass to all fallbacks
llm_metadata: Additional metadata including region_name for Bedrock
Returns:
List of fallback configurations, each containing:
- provider: str
- model: str
- retry: int (0 = no retry, >0 = retry count)
- system_instruction: str
- region_name: str (for Bedrock only)
"""
provider = (provider or "").lower()
fallbacks = []
# Provider-specific fallback chains
if provider == "gemini":
if model_name == "gemini-2.5-pro":
fallbacks.append({"provider": "gemini", "model": "gemini-2.5-flash", "retry": 0})
else:
fallbacks.append({"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0})
fallbacks.append({"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0})
elif provider == "byteplus":
fallbacks.append({"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0})
elif provider == "bedrock":
if model_name == "deepseek.v3-v1:0":
fallbacks.append({"provider": "byteplus", "model": "deepseek-v3-1-250821", "retry": 0})
fallbacks.append({"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0})
else:
print(f"[LLM_GATEWAY_CLIENT] ⚠️ No specific fallback defined for provider: {provider}")
# Universal fallback (always added)
fallbacks.append({"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0})
# Enrich all fallbacks with system_instruction and region_name
for fb in fallbacks:
fb["system_instruction"] = system_instruction
if fb["provider"] == "bedrock" and llm_metadata:
fb["region_name"] = llm_metadata.get("region_name", "ap-south-1")
return fallbacks
Example 1: Gemini 2.5-Pro Request
fallbacks = get_fallback_routing("gemini", "gemini-2.5-pro", "You are a helpful assistant")
# Result:
[
{"provider": "gemini", "model": "gemini-2.5-flash", "retry": 0, "system_instruction": "..."},
{"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0, "system_instruction": "..."},
{"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0, "system_instruction": "..."}
]
Example 2: AWS Bedrock Deepseek Request
fallbacks = get_fallback_routing(
"bedrock",
"deepseek.v3-v1:0",
"You are an astrologer",
llm_metadata={"region_name": "ap-south-1"}
)
# Result:
[
{"provider": "byteplus", "model": "deepseek-v3-1-250821", "retry": 0, "system_instruction": "..."},
{"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0, "system_instruction": "..."},
{"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0, "system_instruction": "..."}
]
# Pseudo-code for LLM Gateway
def generate_with_fallbacks(primary_config, fallback_chain):
try:
return call_llm(primary_config)
except Exception as e:
log_error(f"Primary provider failed: {e}")
for fallback in fallback_chain:
try:
return call_llm(fallback)
except Exception as fe:
log_error(f"Fallback {fallback['provider']}/{fallback['model']} failed: {fe}")
continue
# All fallbacks exhausted
raise Exception("All LLM providers failed")
retry: 0 means don't retry within fallback, just move to next
</fallback_mechanisms><region_configuration>
# Store region in BotLLMConfig metadata
bot_llm_config = BotLLMConfig.objects.create(
bot=bot,
llm_provider="bedrock",
model_name="deepseek.v3-v1:0",
metadata={
"region_name": "ap-south-1", # AWS region
"additional_config": {...}
}
)
# When building fallback chain
fallbacks = get_fallback_routing(
provider="bedrock",
model_name="deepseek.v3-v1:0",
system_instruction=prompt,
llm_metadata=bot_llm_config.metadata
)
def get_bedrock_region_fallbacks(model_name, regions=["ap-south-1", "us-east-1", "eu-west-1"]):
"""
Create fallback chain across multiple AWS regions.
"""
fallbacks = []
for region in regions:
fallbacks.append({
"provider": "bedrock",
"model": model_name,
"region_name": region,
"retry": 0
})
# Add non-Bedrock fallbacks
fallbacks.append({"provider": "gemini", "model": "gemini-2.0-flash", "retry": 0})
return fallbacks
# Primary region from user location or bot config
primary_region = determine_region_from_user(user) # e.g., "ap-south-1" for India
# Fallback regions based on latency/availability
fallback_regions = ["us-east-1", "eu-west-1"]
# Build full chain
region_chain = [primary_region] + fallback_regions
</region_configuration>
<prompt_templating>
class PromptTemplateHelper:
"""
Helper class for rendering prompt templates by resolving variable placeholders.
"""
@classmethod
def render_template(cls, prompt_text: str, **kwargs) -> str:
"""
Render a prompt template by replacing variable placeholders with actual values.
Example:
prompt = "Hi {{user_name}}, today is {{current_date}}."
result = PromptTemplateHelper.render_template(prompt, user_doc=user_doc)
# Output: "Hi John, today is 15 January 2026."
"""
function_map = cls.get_function_mapping()
# Find all {{variable_name}} in the prompt
variables = re_findall(r"\{\{(.*?)\}\}", prompt_text)
for var in variables:
function = function_map.get(var)
if function:
value = function(**kwargs)
prompt_text = prompt_text.replace(f"{{{{{var}}}}}", str(value or ""))
return prompt_text
AVAILABLE_VARIABLES = [
# Bot-specific variables
{"name": "{{bot_name}}", "description": "Name of the bot", "category": "Bot Info"},
{"name": "{{bot_description}}", "description": "Bot description", "category": "Bot Info"},
{"name": "{{bot_scene_desc}}", "description": "Bot Scene description", "category": "Bot Info"},
# User interaction variables
{"name": "{{user_name}}", "description": "User name", "category": "User Input"},
{"name": "{{user_age}}", "description": "User age", "category": "User Input"},
{"name": "{{user_gender}}", "description": "User gender", "category": "User Input"},
{"name": "{{user_message}}", "description": "Current user message", "category": "User Input"},
{"name": "{{conversation_history}}", "description": "Conversation History", "category": "User Input"},
{"name": "{{user_astro_data}}", "description": "Astro data for the user", "category": "User Input"},
# System variables
{"name": "{{current_time}}", "description": "Current time", "category": "System"},
{"name": "{{current_date}}", "description": "Current date", "category": "System"},
{"name": "{{day_of_week}}", "description": "Current day of week", "category": "System"},
# Common prompt variables
{"name": "{{random_number_1_4}}", "description": "Generate random number 1-4", "category": "Common"},
]
@classmethod
def get_function_mapping(cls):
"""Return mapping of variable names to handler functions."""
return {
# Bot-specific variables
"bot_name": cls.function_bot_name,
"bot_description": cls.function_bot_description,
"bot_scene_desc": cls.function_bot_scene_desc,
# User interaction variables
"user_name": cls.function_user_name,
"user_age": cls.function_user_age,
"user_gender": cls.function_user_gender,
"user_message": cls.function_user_message,
"user_astro_data": cls.function_user_astro_data,
# System variables
"current_time": cls.function_current_time,
"current_date": cls.function_current_date,
"day_of_week": cls.function_day_of_week,
"conversation_history": cls.function_conversation_history,
# Common prompt variables
"random_number_1_4": cls.function_random_number_1_4,
}
@classmethod
def function_current_time(cls, **kwargs):
"""Return the current system time."""
return UtilHelper.timezone_to_ist(datetime.now()).strftime("%I:%M %p")
@classmethod
def function_current_date(cls, **kwargs):
"""Return the current system date."""
return UtilHelper.timezone_to_ist(datetime.now()).strftime("%d %B %Y")
@classmethod
def function_user_name(cls, **kwargs):
"""Return user name from user_doc."""
return cls.get_sanitized_user_doc(**kwargs).get("name", "")
# Define template in database or config
prompt_template = """
You are {{bot_name}}, a helpful assistant.
User Information:
- Name: {{user_name}}
- Age: {{user_age}}
- Gender: {{user_gender}}
Current Context:
- Date: {{current_date}}
- Time: {{current_time}}
- Day: {{day_of_week}}
User Message: {{user_message}}
Please respond helpfully and naturally.
"""
# Render with actual values
rendered_prompt = PromptTemplateHelper.render_template(
prompt_template,
bot_doc={"name": "Mira", "description": "Vedic astrologer"},
user_doc={"name": "John", "age": 28, "gender": "male", "messages": "What's my future?"}
)
# Output:
"""
You are Mira, a helpful assistant.
User Information:
- Name: John
- Age: 28
- Gender: male
Current Context:
- Date: 15 January 2026
- Time: 02:45 PM
- Day: Thursday
User Message: What's my future?
Please respond helpfully and naturally.
"""
</prompt_templating>
<prompt_design_patterns>
@classmethod
def get_mira_prompt(cls, **kwargs):
"""
Build YAML-structured prompt for astrology bot.
Required kwargs: bot_doc, language, app_id
Optional: memories, profile_details_1, profile_details_2
"""
bot_doc = kwargs.get("bot_doc")
_prompt = {
"Role": f"You are {bot_doc.get('name')}, an experienced {str.upper(bot_doc.get('sex', ''))} Vedic astrologer with expertise in predictive astrology...",
"Query House Mapping": {
"Marriage": {
"Houses To Analyze": "7, 2, 11, 8",
"Karaka Planets": "Venus, Jupiter",
"Special Checks": "Manglik status, 7th lord placement, Venus strength, Jupiter aspects",
"Timing Triggers": "7th lord dasha, Venus period, Jupiter transit to 7th",
},
"Career": {
"Houses To Analyze": "10, 1, 2, 6, 11",
"Karaka Planets": "Sun, Saturn, Mercury",
"Special Checks": "10th lord strength, Sun placement, D9 10th house",
"Timing Triggers": "10th lord dasha, Saturn period, Sun antardasha",
},
# ... more query types
},
"Analysis Steps": {
"Core": "Understand the question AND the feeling underneath. Map to houses, planets, timing.",
"Analysis": "Houses → Lords → Karakas → Dashas → Divisionals → Yogas → Doshas.",
"Timing": "Find activation windows. Always check next 60 days for shifts.",
"Response Crafting": "Give analysis and response following guidelines below.",
},
"Response Format": {
"Structure": "A single paragraph text based on the instructions",
"Style": "Insightful, Helpful, focussed, non-repetitive",
<!-- Content truncated for initial SEO render. Open the source file tab for the full file. -->
npx skills add twofourlabs/ai-integrated-api-backend下载完整 Skill 目录，包含 SKILL.md 及所有相关文件
Search for places (restaurants, cafes, etc.) via Google Places API proxy on localhost.
Interact with GitHub using the `gh` CLI. Use `gh issue`, `gh pr`, `gh run`, and `gh api` for issues, PRs, CI runs, and advanced queries.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Start voice calls via the OpenClaw voice-call plugin.
Notion API for creating and managing pages, databases, and blocks.
Gemini CLI for one-shot Q&A, summaries, and generation.
Category:developer