Azure Cosmos DB is a NoSQL database management system. It's compatible with some existing NoSQL systems, including MongoDB and Cassandra. In the Contoso scenario, you've created a Cosmos DB database for holding information about the quantity of items in stock. Now you need to understand how to populate this database and how to query it. Now you'll review how Cosmos DB stores data. Then you'll learn how to upload data to a Cosmos DB database and configure Cosmos DB to support bulk loading. Cosmos DB manages data as a set of documents. It uses JSON or JavaScript Object Notation to represent the document structure. In this format, the fields in a document are enclosed between braces, and each field is prefixed with its name. A document is a collection of fields identified by a key. The fields in each document can vary, and a field can contain child documents. As an example, you can have a pair of documents representing customer information. In both cases, each customer document includes child documents containing the name and address, but the fields in these child documents vary between customers. Documents in a Cosmos DB database are organized into containers. The documents in a container are grouped together into partitions. A partition holds a set of documents that share a common partition key. You designate one of the fields in your documents as the partition key. You should select a partition key that collects all related documents together. This approach helps to reduce the amount of I/O disk reads that queries might need to perform when retrieving a set of documents for a given entity. For example, in a document database for an e-commerce system recording the details of costumers and the orders they've placed, you can partition the data by customer ID and store the customer and order details for each customer in the same partition. To find all the information and orders for a customer, you simply need to query that single partition. Cosmos DB is a foundational service in Microsoft Azure and has been used by many Microsoft products for mission critical applications at global scale, including Skype, Xbox, Microsoft 365, and Microsoft Azure. Cosmos DB is highly suitable for IoT and telematics, retail and marketing, gaming, and web and mobile applications. For additional information about uses for Cosmos DB, read common Azure Cosmos DB use cases, which is linked to in the additional readings at the end of this lesson. You access the data in a Cosmos DB database through a set of commands and operations, collectively known as an API. API stands for Application Programming Interface. Here are some APIs that Cosmos DB currently supports. Cosmos DB provides its own native API called the SQL API. This API provides a SQL like query language over documents that enables you to retrieve documents using select statements. For example, to retrieve the address for a specific customer whose details are stored in the costumer container, you can select the address field in the costumer container, where the customer ID is equal to your desired search term. Cosmos DB also provides other APIs that enable you to access these documents using the command sets of other NoSQL database management systems. These other APIs are, the Table API interface, which enables you to use the Azure Table Storage API to store and retrieve documents. The purpose of this interface is to enable you to switch from Table Storage to Cosmos DB without requiring that you modify your existing applications. MongoDB is another well-known document database with its own programmatic interface. Many organizations run MongoDB on premises. You can use the MongoDB API for Cosmos DB to enable a MongoDB application to run unchanged against a Cosmos DB database. You can migrate the data in the MongoDB database to Cosmos DB running in the Cloud, but continue to run your existing applications to access this data. Cassandra is a column family database management system. This is another database management system that many organizations run on premises. The Cassandra API for Cosmos DB provides a Cassandra like programmatic interface for Cosmos DB. Cassandra API requests are mapped to Cosmos DB document requests. As with the MongoDB API, the primary purpose of the Cassandra API is to enable you to quickly migrate Cassandra databases and applications to Cosmos DB. Finally, the Gremlin API implements a graph database interface to Cosmos DB. A graph is a collection of data objects and directed relationships. Data is still held as a set of documents in Cosmos DB, but the Gremlin API enables you to perform graph queries over data. Using the Gremlin API, you can walk through the objects and relationships in the graph that scope for all manner of complex relationships, such as what is the name of the pet of Sam's landlord? The principal use of the Table, MongoDB, and Cassandra APIs is to support existing applications written using these data stores. If you're building a new application and database, you should use the SQL API or Gremlin API. Cosmos DB provides several options for uploading data to a Cosmos DB database and querying that data. You can use Data Explorer in the Azure portal to run ad hoc queries. You can also use this tool to load data, but you can only load one document at a time. The data load functionality is primarily aimed at uploading a small number of documents, up to two megabytes in total size for test purposes, rather than importing large quantities of data. You can also use the Cosmos DB data Migration tool to perform a bulk load or transfer of data from another data source. Additionally, you can use Azure Data Factory to import data from another source. You can write a custom application that imports data using the Cosmos DB BulkExecutor library. This strategy is beyond the scope of this module. Finally, you can create your own application that uses the functions available to the Cosmos DB SQL API client library to store data. This approach is also beyond the scope of this module. You can use the Data Migration tool to import data to Azure Cosmos DB from a variety of sources, including: JSON files, MongoDB, SQL Server, CSV files, Azure Table Storage, Amazon Dynamo DB, HBase, and Azure Cosmos containers. The Data Migration tool is available as a download from GitHub. The tool guides you through the process of migrating data into a Cosmos DB database. You're prompted for the source of the data, one of the items listed earlier, and the destination, the Cosmos DB database and container. The tool can either populate an existing container or create a new one if the specified container doesn't already exist. You should note that you can also use the Data Migration tool to export data from a Cosmos DB container to a JSON file, either held locally or in Azure Blob storage. If you have a large amount of data, the Data Migration tool can make use of multiple concurrent threads to batch your data into chunks and load the chunks in parallel. Each thread acts as a separate client connection to the database. Bulk loading can become a write intensive task. When you upload data to a container, if you have insufficient throughput capacity configured to support the volume of write operations occurring concurrently, some of the upload requests will fail. Cosmos DB reports a HTTP 429 error, request rate is large. Therefore, if you're planning on performing a large data import, you should increase the throughput resources available to the target Cosmos container. If you're using the Data Migration tool to create the container as well as populate it, the target information page enables you to specify the throughput resources to allocate. If you've already created the container, use the scale settings of the database in the Data Explorer page for your database in the Azure portal to specify the maximum throughput, or set the throughput to auto-scale. Once the data has been loaded, you may be able to reduce the throughput resources to lower the cost of the database. Now that you've learned how you can import and export data to and from an Azure Cosmos DB database, let's see how you can query a Cosmos DB database.
