Building a Sports Data API with MongoDB and Flask

In this project, we set out to create a flexible, high-performance sports data API to manage complex data on football teams, players, matches, and competitions. Using the Data Sports Group API as our data source, we developed a process to import this data into MongoDB, ensuring efficient storage and quick access. We then built a custom Flask API to provide endpoints that deliver this data to applications, making the data easy to retrieve and integrate. Here’s an overview of how we approached this project, including a detailed example to illustrate the process.

If you’d like to get started right away, read Setting Up MongoDB with Python for Data Import and API Access

Step 1: Data Import from the Data Sports Group API

We began by writing scripts to fetch data from the Data Sports Group API. The data, provided in JSON format, included details about players, teams, squads, and other football-related information. Here’s a look at the main steps:

1. API Requests: Using the requests library in Python, we pulled data from various endpoints, such as those for team details, player statistics, and match information.
2. MongoDB Collections: The data was organized into multiple MongoDB collections:
   • people: Holds detailed data about players, indexed by fields such as people_id and membership_id.
   • team: Stores team details, including unique identifiers and team metadata.
   • squad: A streamlined collection containing team summaries and lists of players within each team.
3. Field Management: We carefully structured fields in MongoDB, using indexes to improve search speed and controlling field types to maintain data consistency. For example, fields such as goals were kept as strings to match the original format, avoiding issues with data conversion.

Step 2: Building the Flask API

To provide easy access to our sports data, we developed a Flask API that served as a direct interface with MongoDB. This API allowed for flexible queries, enabling users to retrieve only the data they needed. Key features of the Flask API included:

1. Flexible Endpoints: The API included multiple endpoints, allowing access to player, team, and competition information. For example, the /team/<team_id>/players endpoint allows users to retrieve all players associated with a specific team, optionally filtering by player statistics or career details.
2. Case-Insensitive Queries: We implemented case-insensitive search functionality to improve user experience. For example, searching for a player’s common_name would return the correct result regardless of capitalization.
3. Optimized JSON Output: By setting ensure_ascii=False in json.dumps(), we ensured that JSON responses were readable without unnecessary Unicode escaping.
4. Database Configuration: The database connection was abstracted in a separate file, db.py, where the get_db() function managed MongoDB connections. This design streamlined the code and made it easy to modify database configurations without changing core API logic.

Example: Fetching Players from a Team

Here’s an example of how our system works. Suppose we want to retrieve a list of players from a specific team. With our Flask API, we can accomplish this through the following endpoint:

Copied!GET /team/<team_id>/players</team_id>
GET /team/<team_id>/players

Request Example

A request to /team/123/players would trigger the following sequence:

1. The Flask API retrieves the data from the squad collection based on the provided team_id.
2. The data is filtered to return only relevant player IDs.
3. Using these IDs, detailed player information is retrieved from the people collection.

Response Example

The response would be a structured JSON output, as shown below:

Copied!{
  "team_id": "123",
  "team_name": "Example FC",
  "players": [
    {
      "people_id": "456",
      "common_name": "John Doe",
      "position": "Forward",
      "goals": "10"
    },
    {
      "people_id": "789",
      "common_name": "Jane Smith",
      "position": "Midfielder",
      "goals": "5"
    }
  ]
}
{
  "team_id": "123",
  "team_name": "Example FC",
  "players": [
    {
      "people_id": "456",
      "common_name": "John Doe",
      "position": "Forward",
      "goals": "10"
    },
    {
      "people_id": "789",
      "common_name": "Jane Smith",
      "position": "Midfielder",
      "goals": "5"
    }
  ]
}

This example demonstrates how the API provides quick, flexible access to team and player data, making it ideal for applications that require real-time or on-demand sports data.

Conclusion

This project showcases the power of MongoDB and Flask when managing and serving complex, large-scale sports data. By combining robust data storage with a flexible API, we built an efficient system that enables fast access to detailed football information. This project is a great example of how modern data tools can simplify complex data management, offering both scalability and usability in one package.