Amazon Elasticsearch Service
开发人员指南 (API 版本 2015-01-01)
AWS 文档中描述的 AWS 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅中国的 AWS 服务入门

签署对 Amazon Elasticsearch Service 的 HTTP 请求

本章包含如何使用 Elasticsearch 客户端和其他常见库将已签名 HTTP 请求发送到 Amazon Elasticsearch Service 的示例。这些代码示例用于与 Elasticsearch API(如 _index_bulk_snapshot)进行交互。如果您的域访问策略包含 IAM 用户或角色,您必须签署针对 Elasticsearch API 的请求。

重要

有关如何与配置 API 进行交互(包括创建、更新和删除 Amazon ES 域等操作)的示例,请参阅在 Amazon Elasticsearch Service 中使用 AWS 开发工具包

主题

Java

发送已签名请求的最简单的方式是使用 AWS 请求签名拦截程序。该存储库包含一些可帮助您入门的示例,或者您可以在 GitHub 上下载 Amazon ES 的示例项目

以下示例使用 Elasticsearch 低级别 Java REST 客户端执行两个不相关的操作:注册快照存储库和对文档进行索引。必须提供 regionhost 的值。

import org.apache.http.HttpEntity; import org.apache.http.HttpHost; import org.apache.http.HttpRequestInterceptor; import org.apache.http.entity.ContentType; import org.apache.http.nio.entity.NStringEntity; import org.elasticsearch.client.Request; import org.elasticsearch.client.Response; import org.elasticsearch.client.RestClient; import com.amazonaws.auth.AWS4Signer; import com.amazonaws.auth.AWSCredentialsProvider; import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; import com.amazonaws.http.AWSRequestSigningApacheInterceptor; import java.io.IOException; public class AmazonElasticsearchServiceSample { private static String serviceName = "es"; private static String region = "us-west-1"; private static String aesEndpoint = "https://domain.us-west-1.es.amazonaws.com"; private static String payload = "{ \"type\": \"s3\", \"settings\": { \"bucket\": \"your-bucket\", \"region\": \"us-west-1\", \"role_arn\": \"arn:aws:iam::123456789012:role/TheServiceRole\" } }"; private static String snapshotPath = "/_snapshot/my-snapshot-repo"; private static String sampleDocument = "{" + "\"title\":\"Walk the Line\"," + "\"director\":\"James Mangold\"," + "\"year\":\"2005\"}"; private static String indexingPath = "/my-index/_doc"; static final AWSCredentialsProvider credentialsProvider = new DefaultAWSCredentialsProviderChain(); public static void main(String[] args) throws IOException { RestClient esClient = esClient(serviceName, region); // Register a snapshot repository HttpEntity entity = new NStringEntity(payload, ContentType.APPLICATION_JSON); Request request = new Request("PUT", snapshotPath); request.setEntity(entity); // request.addParameter(name, value); // optional parameters Response response = esClient.performRequest(request); System.out.println(response.toString()); // Index a document entity = new NStringEntity(sampleDocument, ContentType.APPLICATION_JSON); String id = "1"; request = new Request("PUT", indexingPath + "/" + id); request.setEntity(entity); // Using a String instead of an HttpEntity sets Content-Type to application/json automatically. // request.setJsonEntity(sampleDocument); response = esClient.performRequest(request); System.out.println(response.toString()); } // Adds the interceptor to the ES REST client public static RestClient esClient(String serviceName, String region) { AWS4Signer signer = new AWS4Signer(); signer.setServiceName(serviceName); signer.setRegionName(region); HttpRequestInterceptor interceptor = new AWSRequestSigningApacheInterceptor(serviceName, signer, credentialsProvider); return RestClient.builder(HttpHost.create(aesEndpoint)).setHttpClientConfigCallback(hacb -> hacb.addInterceptorLast(interceptor)).build(); } }

如果您更喜欢高级别 REST 客户端(提供了大多数相同的功能和更简单的代码),请尝试以下示例,该示例还使用了 AWS 请求签名拦截程序

import org.apache.http.HttpHost; import org.apache.http.HttpRequestInterceptor; import org.elasticsearch.action.index.IndexRequest; import org.elasticsearch.action.index.IndexResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestHighLevelClient; import com.amazonaws.auth.AWS4Signer; import com.amazonaws.auth.AWSCredentialsProvider; import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; import com.amazonaws.http.AWSRequestSigningApacheInterceptor; import java.io.IOException; import java.util.HashMap; import java.util.Map; public class AmazonElasticsearchServiceSample { private static String serviceName = "es"; private static String region = "us-west-1"; private static String aesEndpoint = ""; // e.g. https://search-mydomain.us-west-1.es.amazonaws.com private static String index = "my-index"; private static String type = "_doc"; private static String id = "1"; static final AWSCredentialsProvider credentialsProvider = new DefaultAWSCredentialsProviderChain(); public static void main(String[] args) throws IOException { RestHighLevelClient esClient = esClient(serviceName, region); // Create the document as a hash map Map<String, Object> document = new HashMap<>(); document.put("title", "Walk the Line"); document.put("director", "James Mangold"); document.put("year", "2005"); // Form the indexing request, send it, and print the response IndexRequest request = new IndexRequest(index, type, id).source(document); IndexResponse response = esClient.index(request, RequestOptions.DEFAULT); System.out.println(response.toString()); } // Adds the interceptor to the ES REST client public static RestHighLevelClient esClient(String serviceName, String region) { AWS4Signer signer = new AWS4Signer(); signer.setServiceName(serviceName); signer.setRegionName(region); HttpRequestInterceptor interceptor = new AWSRequestSigningApacheInterceptor(serviceName, signer, credentialsProvider); return new RestHighLevelClient(RestClient.builder(HttpHost.create(aesEndpoint)).setHttpClientConfigCallback(hacb -> hacb.addInterceptorLast(interceptor))); } }

提示

两个已签名示例都使用了默认凭证链。使用 AWS CLI 运行 aws configure 以设置您的凭证。

Python

您可以使用 pip 安装 elasticsearch-py,即适用于 Python 的 Elasticsearch 客户端。您可能更喜欢请求而非使用客户端。requests-aws4auth适用于 Python 的 开发工具包 (Boto3) 程序包简化了身份验证过程,但并非硬性要求。从终端设备运行以下命令:

pip install boto3 pip install elasticsearch pip install requests pip install requests-aws4auth

以下示例代码将建立与指定 Amazon ES 域的安全连接,并使用 _index API 为单个文档编制索引。必须提供 regionhost 的值。

from elasticsearch import Elasticsearch, RequestsHttpConnection from requests_aws4auth import AWS4Auth import boto3 host = '' # For example, my-test-domain.us-east-1.es.amazonaws.com region = '' # e.g. us-west-1 service = 'es' credentials = boto3.Session().get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service) es = Elasticsearch( hosts = [{'host': host, 'port': 443}], http_auth = awsauth, use_ssl = True, verify_certs = True, connection_class = RequestsHttpConnection ) document = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" } es.index(index="movies", doc_type="_doc", id="5", body=document) print(es.get(index="movies", doc_type="_doc", id="5"))

如果您不想使用 elasticsearch-py,则可以仅发出标准 HTTP 请求。此示例将创建一个具有 7 个分片和 2 个副本的新索引:

from requests_aws4auth import AWS4Auth import boto3 import requests host = '' # The domain with https:// and trailing slash. For example, https://my-test-domain.us-east-1.es.amazonaws.com/ path = 'my-index' # the Elasticsearch API endpoint region = '' # For example, us-west-1 service = 'es' credentials = boto3.Session().get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service) url = host + path # The JSON body to accompany the request (if necessary) payload = { "settings" : { "number_of_shards" : 7, "number_of_replicas" : 2 } } r = requests.put(url, auth=awsauth, json=payload) # requests.get, post, and delete have similar syntax print(r.text)

下一示例使用 Beautiful Soup 库帮助基于 HTML 文件本地目录构建批量文件。使用与第一个示例相同的客户端,可以将该文件发送到 _bulk API,以进行索引编制。您可以基于这些代码向网站添加搜索功能:

from bs4 import BeautifulSoup from elasticsearch import Elasticsearch, RequestsHttpConnection from requests_aws4auth import AWS4Auth import boto3 import glob import json bulk_file = '' id = 1 # This loop iterates through all HTML files in the current directory and # indexes two things: the contents of the first h1 tag and all other text. for html_file in glob.glob('*.htm'): with open(html_file) as f: soup = BeautifulSoup(f, 'html.parser') title = soup.h1.string body = soup.get_text(" ", strip=True) # If get_text() is too noisy, you can do further processing on the string. index = { 'title': title, 'body': body, 'link': html_file } # If running this script on a website, you probably need to prepend the URL and path to html_file. # The action_and_metadata portion of the bulk file bulk_file += '{ "index" : { "_index" : "site", "_type" : "_doc", "_id" : "' + str(id) + '" } }\n' # The optional_document portion of the bulk file bulk_file += json.dumps(index) + '\n' id += 1 host = '' # For example, my-test-domain.us-east-1.es.amazonaws.com region = '' # e.g. us-west-1 service = 'es' credentials = boto3.Session().get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service) es = Elasticsearch( hosts = [{'host': host, 'port': 443}], http_auth = awsauth, use_ssl = True, verify_certs = True, connection_class = RequestsHttpConnection ) es.bulk(bulk_file) print(es.search(q='some test query'))

Ruby

第一个示例使用 Elasticsearch Ruby 客户端和 Faraday 中间件执行请求签名。从终端设备运行以下命令:

gem install elasticsearch gem install faraday_middleware-aws-sigv4

此示例代码创建一个新的 Elasticsearch 客户端,将 Faraday 中间件配置为签署请求和为单个文档编制索引。必须提供 hostregion 的值。

require 'elasticsearch' require 'faraday_middleware/aws_sigv4' host = '' # e.g. https://my-domain.region.es.com index = 'ruby-index' type = '_doc' id = '1' document = { year: 2007, title: '5 Centimeters per Second', info: { plot: 'Told in three interconnected segments, we follow a young man named Takaki through his life.', rating: 7.7 } } region = '' # e.g. us-west-1 service = 'es' client = Elasticsearch::Client.new(url: host) do |f| f.request :aws_sigv4, service: service, region: region, access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'], session_token: ENV['AWS_SESSION_TOKEN'] # optional end puts client.index index: index, type: type, id: id, body: document

如果凭证不起作用,则在终端使用以下命令将其导出:

export AWS_ACCESS_KEY_ID="your-access-key" export AWS_SECRET_ACCESS_KEY="your-secret-key" export AWS_SESSION_TOKEN=""your-session-token"

此示例使用 适用于 Ruby 的 AWS 开发工具包 和标准 Ruby 库发送已签名的 HTTP 请求。就像第一个示例,它将为单个文档编制索引。必须提供 host 和 region 的值。

require 'aws-sdk-elasticsearchservice' host = '' # e.g. https://my-domain.region.es.com index = 'ruby-index' type = '_doc' id = '2' document = { year: 2007, title: '5 Centimeters per Second', info: { plot: 'Told in three interconnected segments, we follow a young man named Takaki through his life.', rating: 7.7 } } service = 'es' region = '' # e.g. us-west-1 signer = Aws::Sigv4::Signer.new( service: service, region: region, access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'], session_token: ENV['AWS_SESSION_TOKEN'] ) signature = signer.sign_request( http_method: 'PUT', url: host + '/' + index + '/' + type + '/' + id, body: document.to_json ) uri = URI(host + '/' + index + '/' + type + '/' + id) Net::HTTP.start(uri.host, uri.port, :use_ssl => true) do |http| request = Net::HTTP::Put.new uri request.body = document.to_json request['Host'] = signature.headers['host'] request['X-Amz-Date'] = signature.headers['x-amz-date'] request['X-Amz-Security-Token'] = signature.headers['x-amz-security-token'] request['X-Amz-Content-Sha256']= signature.headers['x-amz-content-sha256'] request['Authorization'] = signature.headers['authorization'] request['Content-Type'] = 'application/json' response = http.request request puts response.body end

Node

此示例使用 适用于 Node.js 中 JavaScript 的 开发工具包。从终端设备运行以下命令:

npm install aws-sdk

此示例代码为单个文档编制索引。必须提供 regiondomain 的值。

var AWS = require('aws-sdk'); var region = ''; // e.g. us-west-1 var domain = ''; // e.g. search-domain.region.es.amazonaws.com var index = 'node-test'; var type = '_doc'; var id = '1'; var json = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" } indexDocument(json); function indexDocument(document) { var endpoint = new AWS.Endpoint(domain); var request = new AWS.HttpRequest(endpoint, region); request.method = 'PUT'; request.path += index + '/' + type + '/' + id; request.body = JSON.stringify(document); request.headers['host'] = domain; request.headers['Content-Type'] = 'application/json'; // Content-Length is only needed for DELETE requests that include a request // body, but including it for all requests doesn't seem to hurt anything. request.headers["Content-Length"] = request.body.length; var credentials = new AWS.EnvironmentCredentials('AWS'); var signer = new AWS.Signers.V4(request, 'es'); signer.addAuthorization(credentials, new Date()); var client = new AWS.HttpClient(); client.handleRequest(request, null, function(response) { console.log(response.statusCode + ' ' + response.statusMessage); var responseBody = ''; response.on('data', function (chunk) { responseBody += chunk; }); response.on('end', function (chunk) { console.log('Response body: ' + responseBody); }); }, function(error) { console.log('Error: ' + error); }); }

如果凭证不起作用,则在终端使用以下命令将其导出:

export AWS_ACCESS_KEY_ID="your-access-key" export AWS_SECRET_ACCESS_KEY="your-secret-key" export AWS_SESSION_TOKEN=""your-session-token"

本页内容: