Data Sources Integration Guide
Connect your external data sources to enrich your AI agent's knowledge. Integrate product catalogs, knowledge bases, and other data to provide more relevant and helpful responses to your customers.
What are Data Sources?
Data Sources allow you to connect external APIs containing your business data (products, courses, articles, services, etc.) to your Chatplugify AI agent. This enables your AI to provide specific, accurate information about your offerings during customer conversations.
External API Integration
Connect any API that follows our format
Automatic Sync
Hourly synchronization keeps data current
AI Enhancement
AI agents can reference your real data
How It Works
Configure Your API Endpoint
Set up your external API endpoint that returns your data in the required format. Configure authentication headers if needed.
Test Connection
Test the connection to ensure your API is responding correctly and data format is valid.
Automatic Synchronization
Our system automatically syncs your data every hour, calling your API with pagination parameters.
AI Enhancement
Your AI agent can now reference your real data during conversations, providing accurate product information, pricing, and recommendations.
API Requirements
Required Response Format
Your API endpoint must return data in the following JSON format:
{
"items": [
{
"id": "prod_001",
"title": "Premium Wireless Headphones",
"description": "High-quality wireless headphones with noise cancellation",
"category": "Electronics",
"type": "product",
"price": "299.99",
"url": "https://example.com/headphones",
"image": "https://example.com/headphones.jpg",
"metadata": {
"brand": "AudioTech",
"color": "Black",
"weight": "250g",
"rating": 4.5,
"inStock": true
}
},
{
"id": "course_001",
"title": "Complete JavaScript Course",
"description": "Learn JavaScript from basics to advanced concepts",
"category": "Programming",
"type": "course",
"price": "99.99",
"url": "https://example.com/js-course",
"image": "https://example.com/js-course.jpg",
"metadata": {
"duration": "40 hours",
"difficulty": "Beginner",
"instructor": "John Developer",
"certification": true
}
}
],
"hasMore": true,
"nextCursor": "course_001"
}
Required Fields
id
Unique identifier for the itemtitle
Display name of the itemdescription
Detailed descriptioncategory
Item categorytype
Item type (product, course, article, etc.)price
Item price (can be "0" or "Free")url
Direct link to the itemimage
URL to item imageOptional Metadata
The metadata object is optional but highly recommended for better AI responses:
Physical Products:
- • brand, color, size, weight
- • rating, reviews, inStock
- • material, dimensions
Courses/Services:
- • duration, difficulty, instructor
- • certification, language
- • location, availability
Pagination Support
Our system calls your API with pagination parameters to handle large datasets efficiently:
How We Call Your API
POST https://your-api.com/items Content-Type: application/json Your-API-Key: your-secret-key { "after": null, // First request "limit": 100 // Items per page } // Subsequent requests: { "after": "item_100_id", // Last item ID from previous response "limit": 100 }
Your Response Format
items
: Array of your items in the required formathasMore
: Boolean indicating if there are more itemsnextCursor
: ID of the last item in current batch (for pagination)
Authentication
Configure custom authentication headers to secure your API endpoint:
Common Header Patterns
x-api-key: abc123def456
Authorization: Bearer token123
api-token: my-secret-token
Security Notes
- • API keys are encrypted at rest
- • Only HTTPS endpoints are supported
- • Workspace isolation ensures data security
Implementation Examples
Node.js / Express Implementation
const express = require('express');
const app = express();
app.use(express.json());
// Your API endpoint that Chatplugify will call
app.post('/api/items', async (req, res) => {
try {
const { after, limit } = req.body;
// Get items from your database
const items = await getItemsFromDatabase(after, limit);
// Check if there are more items
const hasMore = await checkIfMoreItems(items[items.length - 1]?.id);
// Format response according to Chatplugify requirements
const response = {
items: items.map(item => ({
id: item.id,
title: item.title,
description: item.description,
category: item.category,
type: item.type,
price: item.price.toString(),
url: item.url,
image: item.image,
metadata: {
brand: item.brand,
rating: item.rating,
inStock: item.stock > 0,
// Add any other relevant metadata
}
})),
hasMore: hasMore,
nextCursor: items.length > 0 ? items[items.length - 1].id : null
};
res.json(response);
} catch (error) {
res.status(500).json({ error: 'Internal server error' });
}
});
async function getItemsFromDatabase(after, limit) {
// Your database query logic here
// Example with SQL:
let query = 'SELECT * FROM products';
let params = [];
if (after) {
query += ' WHERE id > ?';
params.push(after);
}
query += ' ORDER BY id LIMIT ?';
params.push(limit);
return await db.query(query, params);
}
app.listen(3000, () => {
console.log('API server running on port 3000');
});
Python / Flask Implementation
from flask import Flask, request, jsonify
import sqlite3
app = Flask(__name__)
@app.route('/api/items', methods=['POST'])
def get_items():
try:
data = request.get_json()
after = data.get('after')
limit = data.get('limit', 100)
# Get items from your database
items = get_items_from_database(after, limit)
# Check if there are more items
has_more = check_if_more_items(items[-1]['id'] if items else None)
# Format response according to Chatplugify requirements
response = {
"items": [
{
"id": item['id'],
"title": item['title'],
"description": item['description'],
"category": item['category'],
"type": item['type'],
"price": str(item['price']),
"url": item['url'],
"image": item['image'],
"metadata": {
"brand": item.get('brand'),
"rating": item.get('rating'),
"inStock": item.get('stock', 0) > 0,
# Add any other relevant metadata
}
}
for item in items
],
"hasMore": has_more,
"nextCursor": items[-1]['id'] if items else None
}
return jsonify(response)
except Exception as e:
return jsonify({"error": "Internal server error"}), 500
def get_items_from_database(after, limit):
conn = sqlite3.connect('products.db')
cursor = conn.cursor()
if after:
cursor.execute("""
SELECT * FROM products
WHERE id > ?
ORDER BY id
LIMIT ?
""", (after, limit))
else:
cursor.execute("""
SELECT * FROM products
ORDER BY id
LIMIT ?
""", (limit,))
items = cursor.fetchall()
conn.close()
return [dict(zip([col[0] for col in cursor.description], item)) for item in items]
if __name__ == '__main__':
app.run(debug=True, port=3000)
Best Practices
Data Quality
Ensure your data is well-structured with clear titles and descriptions. The AI performs better with high-quality, detailed information.
Consistent Formatting
Use consistent formatting for prices, categories, and metadata across all items for better AI understanding.
Performance Optimization
Implement efficient pagination and caching to handle large datasets. Response times should be under 5 seconds.
Error Handling
Implement proper error handling and return appropriate HTTP status codes. Failed syncs will be retried automatically.
Troubleshooting
Common Issues
Connection test fails
- • Verify your API endpoint URL is correct and accessible
- • Check authentication headers are properly configured
- • Ensure your API responds to POST requests
- • Verify the response format matches our requirements
Data not syncing
- • Check that required fields (id, title, description) are present
- • Verify pagination parameters are handled correctly
- • Ensure hasMore and nextCursor are returned properly
- • Check server logs for any API errors
Performance issues
- • Optimize database queries for pagination
- • Implement caching for frequently accessed data
- • Consider using database indexes on relevant fields
- • Monitor API response times (should be under 5 seconds)
Ready to Connect Your Data?
Head to your dashboard to set up your first data source and enhance your AI agent's capabilities.