, ,

Setting Up MongoDB with Python for Data Import and API Access

John Timmer Avatar
John Timmer

·

, ,

·

MongoDB offers a flexible, scalable way to store complex datasets, and it pairs well with Python for quick data import and API development. In this guide, we’ll walk through setting up MongoDB, creating a Python script to fetch and store data, building a simple API, and then using PHP to access the data. This setup is particularly useful for dynamic applications that require fast data retrieval and regular updates.

Step 1: Install and Configure MongoDB

Install MongoDB (on Ubuntu as an example):


Copied!

sudo apt update
sudo apt install -y mongodb

Start and Enable MongoDB:


Copied!

sudo systemctl start mongodb
sudo systemctl enable mongodb

Verify Installation: Connect to MongoDB and check the status


Copied!

mongo --eval 'db.runCommand({ connectionStatus: 1 })'

Step 2: Create a Simple Python Script to Import Data

We’ll create a small Python script that retrieves data from an example JSON API and inserts it into MongoDB. Here’s a simple example with sample data.

Install Required Libraries:


Copied!

pip install requests pymongo

Python Script to Fetch and Store Data

Create a file called data_import.py and add the following code:


Copied!

import requests
from pymongo import MongoClient

# Connect to MongoDB
client = MongoClient('mongodb://localhost:27017/')
db = client['sports_data']  # Database name
collection = db['players']  # Collection name

# Fetch data from an external example JSON API
url = 'https://api.example.com/players'
response = requests.get(url)
data = response.json()

# Update or insert data
for player in data:
    collection.update_one(
        {'player_id': player['player_id']},
        {'$set': player},
        upsert=True  # Insert the document if it doesn't exist
    )

print("Data imported successfully.")

Run the Script:


Copied!

python data_import.py

This script connects to MongoDB, retrieves data from an API, and upserts each document (inserts if it doesn’t exist, updates if it does).

Step 3: Build a Python API with Flask

Next, we’ll create a simple Flask API to make our MongoDB data accessible. This API will serve as the middle layer between MongoDB and other applications, like PHP.

Install Flask:


Copied!

pip install flask

Create the Flask API:

Create a file called app.py and add the following code:


Copied!

from flask import Flask, jsonify, request
from pymongo import MongoClient

app = Flask(__name__)

# MongoDB connection
client = MongoClient('mongodb://localhost:27017/')
db = client['sports_data']
collection = db['players']

# Endpoint to retrieve players
@app.route('/players', methods=['GET'])
def get_players():
    players = list(collection.find(Array, {'_id': 0}))  # Exclude MongoDB's _id field
    return jsonify(players)

# Run the Flask app
if __name__ == '__main__':
    app.run(debug=True, port=5000)

Run the Flask API:


Copied!

python app.py

This API will be accessible at http://localhost:5000/players, returning a JSON list of all players in the database.

Step 4: Access the Python API with PHP

Finally, we’ll create a PHP script to fetch data from the Python API. This allows us to use the API in a PHP application and integrate MongoDB data seamlessly.

Create the PHP Script:Create a PHP file called get_players.php and add the following code:


Copied!

<?php
// Set the API URL
$url = 'http://localhost:5000/players';

// Initialize cURL session
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// Execute the cURL request and decode the JSON response
$response = curl_exec($ch);
curl_close($ch);

// Convert JSON response to PHP array
$players = json_decode($response, true);

// Display the players
foreach ($players as $player) {
    echo "Player ID: " . $player['player_id'] . "<br>";
    echo "Name: " . $player['name'] . "<br>";
    echo "Team: " . $player['team'] . "<br><br>";
}
?>

Run the PHP Script: Host this PHP file on a local server (e.g., with XAMPP, WAMP, or using php -S localhost:8000) and access it via http://localhost/get_players.php. This script fetches the JSON data from the Python API and displays it.

Conclusion

With this setup, you now have a full pipeline from MongoDB to Python and PHP. Data can be imported into MongoDB, accessed through a Python API, and used in PHP applications. This setup is versatile and can be scaled or customized for different use cases, making it a great foundation for dynamic data-driven applications.