Azure Search Indexers – Index data without writing code

Azure Search logo big

The Microsoft Azure Search team recently released a new preview version which includes some interesting new features. I’ve briefly covered them in my previous Azure Search Services post but wanted to dive a bit deeper into the Indexers.

An indexer is basically a content crawler written for a specific data source eliminating the need for writing code responsible for extract data from your data source and feed this data within the index. The indexer runs within the Azure context and it’s therefore not required to create a service for hosting this process (billing details are not at available yet).

At the time of writing, it’s only possible to configure a DocumentDB and Azure SQL Database as a data source. However, this might change in the future depending on customer needs.

Azure Search Indexers – Getting started

Before we start creating an indexer, I necessary to understand what’s involved. There are basically 4 parts when setting up an indexer; a supported data source (containing your data), a pointer to your data source (also called data source, storing the connection string), an index and the Indexer.

For now, it isn’t possible to create neither the data source pointer nor the indexer using the Azure portal/PowerShell management API. Therefore it’s required to use the REST based API. I will be using Chrome’s Postman for creating and sending HTTP requests.

API Details can be found here https://msdn.microsoft.com/en-us/library/azure/dn946891.aspx

Test Project Setup

I just wanted to keep everything very simple and therefore configured the following test setup using the Azure Management Portal.

  1. Install Chrome’s Postman (or a similar tool) for creating and sending HTTP requests (getpostman.com)
  2. Create a new Azure SQL Database. I’m using the AdventureWorksLT [V12] sample database for sample data.
    Azure Search Indexers Database
  3. Open the Properties blade of the database, select “Show database connection strings” and copy the ADO.NET version of the connection string
    Azure-Search-Indexers-Database-2
  4. Connect to the database using SQL Server Management Studio and create the following view. This because it’s required having a column named id
    CREATE VIEW [SalesLT].[vProductAndDescriptionIndex] 
    WITH SCHEMABINDING 
    AS 
    -- View (indexed or standard) to display products and product descriptions by language.
    SELECT 
        p.[ProductID] as [id]
        ,p.[Name] 
        ,pm.[Name] AS [ProductModel] 
        ,pmx.[Culture] 
        ,pd.[Description] 
    FROM [SalesLT].[Product] p 
        INNER JOIN [SalesLT].[ProductModel] pm 
        ON p.[ProductModelID] = pm.[ProductModelID] 
        INNER JOIN [SalesLT].[ProductModelProductDescription] pmx 
        ON pm.[ProductModelID] = pmx.[ProductModelID] 
        INNER JOIN [SalesLT].[ProductDescription] pd 
        ON pmx.[ProductDescriptionID] = pd.[ProductDescriptionID];
    GO
    
  5. Create a new Azure Search service
  6. After completion, open the Search Service Essentials Blade and copy one of the Administration keys.
    Azure Search Indexers Key
  7. Create a new Index (I’ve named mine index1) and add the following fields
    Name: String, Searchable, Retrievable
    Description: String, Searchable, Retrievable
  8. Azure Search Indexers Index Fields

Creating a new Search Service Data Source

Open Postman and make sure to alter the following:

  1. Change the type to POST
  2. Provide the URL pointing to your Search Service endpoint including the api-version as shown below https://[servicename].search.windows.net/datasources?api-version=2015-02-28-Preview
  3. Within the headers section add Content-Type : application/json and api-key : [your admin key]
  4. Provide the following message body however make sure to alter the connection string
{
"name" : "datasource1",
"type" : "azuresql",
"credentials" : { "connectionString" : "Server=tcp:azuresearchindexer.database.windows.net,1433;Database=AzureSearchIndexer;User ID=XYZ;XYZ @A;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;" },
"container" : { "name" : "[SalesLT].[vProductAndDescriptionIndex] " }
}

The container value [SalesLT].[vProductAndDescriptionIndex] points to the view created earlier.

Azure Search Indexers Data Source

Creating a new Search Service Indexer

Open Postman and make sure to alter the following:

  1. Change the type to POST
  2. Provide the URL pointing to your Search Service endpoint including the api-version as shown below https://[servicename].search.windows.net/indexers?api-version=2015-02-28-Preview
  3. Within the headers section add Content-Type : application/json and api-key : [your admin key]
{ 
"name" : "indexer1",
"dataSourceName" : "datasource1",
"targetIndexName" : "index1"
}

In this example index1 refers to the name of the search Index created earlier.

Run the indexer and retrieve indexer status information

Within Postman and make sure to alter the following:

  1. Change the type to POST
  2. Provide the URL pointing to your Search Service endpoint including the api-version and name of the indexer as shown below
    https://[servicename].search.windows.net/indexers/indexer1/run?api-version=2015-02-28-Preview
  3. Within the headers section add api-key : [your admin key]

The Get Indexer Status operation retrieves the current status and execution history of an indexer:

  1. Change the type to POST
  2. Provide the URL pointing to your Search Service endpoint including the api-version and name of the indexer as shown below
    https://[servicename].search.windows.net/indexers/[indexer name]/status?api-version=2015-02-28-Preview
  3. Within the headers section add api-key : [your admin key]

Testing the index

The following URL template can be used to test the index. Just keep in mind that it might take some time before the data has been indexed, depending on the amount of data and the selected tier.

https:// [servicename].search.windows.net/indexes/index1/docs?search=finger&api-version=2015-02-28

"value": [
{
"@search.score": 1.3257511,
"id": "858",
"Name": "Half-Finger Gloves, S"
},
{
"@search.score": 1.3257511,
"id": "859",
"Name": "Half-Finger Gloves, M"
},
{
"@search.score": 1.3257511,
"id": "860",
"Name": "Half-Finger Gloves, L"
},
....

As you can see, it’s really simple to get started with Azure Search. Just keep in mind that I haven’t covered options like Data Change and Deletion Detection Policies, indexing schedules and parameters, resetting an indexer and Mapping constraints. However the detailed documentation makes it easy to explore on your own.

Post Navigation