向 Amazon OpenSearch Service 签署 HTTP 请求 - Amazon Opensearch Service
Amazon Web Services 文档中描述的 Amazon Web Services 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅中国的 Amazon Web Services 服务入门

向 Amazon OpenSearch Service 签署 HTTP 请求

这部分包含关于如何使用 Elasticsearch 和 OpenSearch 客户端和其他常见库将已签名 HTTP 请求发送到 Amazon OpenSearch Service 的示例。这些代码示例用于与 OpenSearch API进行交互,如 _index_bulk_snapshot。如果您的域访问策略包含 IAM 用户或角色(或者您使用具有精细访问控制的 IAM 主用户),必须使用 IAM 凭证对 OpenSearch API 请求进行签署。

有关如何与配置 API 进行交互(包括创建、更新和删除 OpenSearch Service 域等操作)的示例,请参阅 使用 Amazon SDK 与 Amazon OpenSearch Service 进行交互

重要

最新版本的 Elasticsearch 客户端可能包括许可证或版本检查,这些检查会人为破坏兼容性。有关使用的正确客户端版本,请参阅 Elasticsearch 客户端兼容性

Java

使用 Java 发送已签名请求的最简单方法是使用 opensearch-java 版本 2.1.0 引入的 AwsSdk2Transport。以下示例显示创建索引,写入文档,然后删除索引。必须提供 regionhost 的值。

package com.amazonaws.samples; import java.io.IOException; import java.util.HashMap; import java.util.Map; import org.opensearch.client.opensearch.OpenSearchClient; import org.opensearch.client.opensearch.core.IndexRequest; import org.opensearch.client.opensearch.indices.CreateIndexRequest; import org.opensearch.client.opensearch.indices.DeleteIndexRequest; import org.opensearch.client.transport.aws.AwsSdk2Transport; import org.opensearch.client.transport.aws.AwsSdk2TransportOptions; import software.amazon.awssdk.http.SdkHttpClient; import software.amazon.awssdk.http.apache.ApacheHttpClient; import software.amazon.awssdk.regions.Region; public class IndexDocument { private static final String host = "search-....us-west-2.es.amazonaws.com"; private static Region region = Region.US_WEST_2; public static void main(String[] args) throws IOException, InterruptedException { SdkHttpClient httpClient = ApacheHttpClient.builder().build(); try { OpenSearchClient client = new OpenSearchClient( new AwsSdk2Transport( httpClient, host, region, AwsSdk2TransportOptions.builder().build())); // create the index String index = "sample-index"; CreateIndexRequest createIndexRequest = new CreateIndexRequest.Builder().index(index).build(); client.indices().create(createIndexRequest); // index data Map<String, Object> document = new HashMap<>(); document.put("firstName", "Michael"); document.put("lastName", "Douglas"); IndexRequest documentIndexRequest = new IndexRequest.Builder() .index(index) .id("2") .document(document) .build(); client.index(documentIndexRequest); // delete the index DeleteIndexRequest deleteRequest = new DeleteIndexRequest.Builder().index(index).build(); client.indices().delete(deleteRequest); } finally { httpClient.close(); } } }

其他替代方法包括使用 Amazon 请求签名拦截程序和/或高级别 REST 客户端。请参阅此示例

提示

此示例使用默认凭证链。使用 Amazon CLI 运行 aws configure 以设置您的凭证。

Python

此示例使用适用于 Python 的 opensearch-py 客户端,您可以使用 pip 安装该客户端。必须提供 regionhost 的值。

from opensearchpy import OpenSearch, RequestsHttpConnection, AWSV4SignerAuth import boto3 host = '' # cluster endpoint, for example: my-test-domain.us-east-1.es.amazonaws.com region = '' # e.g. us-west-1 credentials = boto3.Session().get_credentials() auth = AWSV4SignerAuth(credentials, region) index_name = 'movies' client = OpenSearch( hosts = [{'host': host, 'port': 443}], http_auth = auth, use_ssl = True, verify_certs = True, connection_class = RequestsHttpConnection ) q = 'miller' query = { 'size': 5, 'query': { 'multi_match': { 'query': q, 'fields': ['title^2', 'director'] } } } response = client.search( body = query, index = index_name ) print('\nSearch results:') print(response)

您可能更喜欢请求而非使用客户端。requests-aws4authSDK for Python (Boto3) 程序包简化了身份验证过程,但并非硬性要求。从终端设备运行以下命令:

pip install boto3 pip install opensearch-py pip install requests pip install requests-aws4auth

以下示例代码将建立与指定 OpenSearch Service 域的安全连接,并为单个文档编制索引。必须提供 regionhost 的值。

from opensearchpy import OpenSearch, RequestsHttpConnection from requests_aws4auth import AWS4Auth import boto3 host = '' # For example, my-test-domain.us-east-1.es.amazonaws.com region = '' # e.g. us-west-1 service = 'es' credentials = boto3.Session().get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token) search = OpenSearch( hosts = [{'host': host, 'port': 443}], http_auth = awsauth, use_ssl = True, verify_certs = True, connection_class = RequestsHttpConnection ) document = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" } search.index(index="movies", doc_type="_doc", id="5", body=document) print(search.get(index="movies", doc_type="_doc", id="5"))

如果您不想使用 opensearch-py,则可以仅发出标准 HTTP 请求。此示例将创建一个具有 7 个分片和 2 个副本的新索引:

from requests_aws4auth import AWS4Auth import boto3 import requests host = '' # The domain with https:// and trailing slash. For example, https://my-test-domain.us-east-1.es.amazonaws.com/ path = 'my-index' # the OpenSearch API endpoint region = '' # For example, us-west-1 service = 'es' credentials = boto3.Session().get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token) url = host + path # The JSON body to accompany the request (if necessary) payload = { "settings" : { "number_of_shards" : 7, "number_of_replicas" : 2 } } r = requests.put(url, auth=awsauth, json=payload) # requests.get, post, and delete have similar syntax print(r.text)

您可以使用自动刷新凭证(而不是静态凭证)构建一个 AWS4Auth 实例,该实例适用于使用 AssumeRole 长时间运行的应用程序。可刷新的凭证实例用于为每个请求生成有效的静态凭证,从而无需在临时证书过期时重新创建 AWS4Auth 实例:

from requests_aws4auth import AWS4Auth from botocore.session import Session credentials = Session().get_credentials() auth = AWS4Auth(region=us-west-1', service='es', refreshable_credentials=credentials)

下一示例使用 Beautiful Soup 库帮助基于 HTML 文件本地目录构建批量文件。使用与第一个示例相同的客户端,可以将该文件发送到 _bulk API,以进行索引编制。您可以基于这些代码向网站添加搜索功能:

from bs4 import BeautifulSoup from opensearchpy import OpenSearch, RequestsHttpConnection from requests_aws4auth import AWS4Auth import boto3 import glob import json bulk_file = '' id = 1 # This loop iterates through all HTML files in the current directory and # indexes two things: the contents of the first h1 tag and all other text. for html_file in glob.glob('*.htm'): with open(html_file) as f: soup = BeautifulSoup(f, 'html.parser') title = soup.h1.string body = soup.get_text(" ", strip=True) # If get_text() is too noisy, you can do further processing on the string. index = { 'title': title, 'body': body, 'link': html_file } # If running this script on a website, you probably need to prepend the URL and path to html_file. # The action_and_metadata portion of the bulk file bulk_file += '{ "index" : { "_index" : "site", "_type" : "_doc", "_id" : "' + str(id) + '" } }\n' # The optional_document portion of the bulk file bulk_file += json.dumps(index) + '\n' id += 1 host = '' # For example, my-test-domain.us-east-1.es.amazonaws.com region = '' # e.g. us-west-1 service = 'es' credentials = boto3.Session().get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service) search = OpenSearch( hosts = [{'host': host, 'port': 443}], http_auth = awsauth, use_ssl = True, verify_certs = True, connection_class = RequestsHttpConnection ) search.bulk(bulk_file) print(search.search(q='some test query'))

Ruby

第一个示例使用 Elasticsearch Ruby 客户端和 Faraday 中间件执行请求签名。请注意,最新版本的客户端可能包含许可证或版本检查,这些检查会人为破坏兼容性。有关使用的正确客户端版本,请参阅 Elasticsearch 客户端兼容性。本示例使用推荐版本 7.13.3。

从终端设备运行以下命令:

gem install elasticsearch -v 7.13.3 gem install faraday_middleware-aws-sigv4

此示例代码将创建一个新的客户端,配置 Faraday 中间件以签署请求,并将为单个文档编制索引。必须提供 full_url_and_portregion 的值。

require 'elasticsearch' require 'faraday_middleware/aws_sigv4' full_url_and_port = '' # e.g. https://my-domain.region.es.amazonaws.com:443 index = 'ruby-index' type = '_doc' id = '1' document = { year: 2007, title: '5 Centimeters per Second', info: { plot: 'Told in three interconnected segments, we follow a young man named Takaki through his life.', rating: 7.7 } } region = '' # e.g. us-west-1 service = 'es' client = Elasticsearch::Client.new(url: full_url_and_port) do |f| f.request :aws_sigv4, service: service, region: region, access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'], session_token: ENV['AWS_SESSION_TOKEN'] # optional end puts client.index index: index, type: type, id: id, body: document

如果凭证不起作用,则在终端使用以下命令将其导出:

export AWS_ACCESS_KEY_ID="your-access-key" export AWS_SECRET_ACCESS_KEY="your-secret-key" export AWS_SESSION_TOKEN="your-session-token"

此示例使用 Amazon SDK for Ruby 和标准 Ruby 库发送已签名的 HTTP 请求。就像第一个示例,它将为单个文档编制索引。必须提供 host 和 region 的值。

require 'aws-sdk-opensearchservice' host = '' # e.g. https://my-domain.region.es.amazonaws.com index = 'ruby-index' type = '_doc' id = '2' document = { year: 2007, title: '5 Centimeters per Second', info: { plot: 'Told in three interconnected segments, we follow a young man named Takaki through his life.', rating: 7.7 } } service = 'es' region = '' # e.g. us-west-1 signer = Aws::Sigv4::Signer.new( service: service, region: region, access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'], session_token: ENV['AWS_SESSION_TOKEN'] ) signature = signer.sign_request( http_method: 'PUT', url: host + '/' + index + '/' + type + '/' + id, body: document.to_json ) uri = URI(host + '/' + index + '/' + type + '/' + id) Net::HTTP.start(uri.host, uri.port, :use_ssl => true) do |http| request = Net::HTTP::Put.new uri request.body = document.to_json request['Host'] = signature.headers['host'] request['X-Amz-Date'] = signature.headers['x-amz-date'] request['X-Amz-Security-Token'] = signature.headers['x-amz-security-token'] request['X-Amz-Content-Sha256']= signature.headers['x-amz-content-sha256'] request['Authorization'] = signature.headers['authorization'] request['Content-Type'] = 'application/json' response = http.request request puts response.body end

Node

此示例使用适用于 JavaScript 的 opensearch-js 客户端来创建索引并添加单个文档。为了签署请求,它会首先使用适用于 JavaScript in Node.js 的 SDK 版本 3 中的 credential-provider-node 模块找到凭证。然后调用 aws4 以使用签名版本 4 签署请求。您必须为 host 提供一个值。

const { Client, Connection } = require("@opensearch-project/opensearch"); const { defaultProvider } = require("@aws-sdk/credential-provider-node"); const aws4 = require("aws4"); var host = '' // e.g. https://my-domain.region.es.amazonaws.com const createAwsConnector = (credentials, region) => { class AmazonConnection extends Connection { buildRequestObject(params) { const request = super.buildRequestObject(params); request.service = 'es'; request.region = region; request.headers = request.headers || {}; request.headers['host'] = request.hostname; return aws4.sign(request, credentials); } } return { Connection: AmazonConnection }; }; const getClient = async () => { const credentials = await defaultProvider()(); return new Client({ ...createAwsConnector(credentials, 'us-east-1'), node: host, }); } async function search() { // Initialize the client. var client = await getClient(); // Create an index. var index_name = "test-index"; var response = await client.indices.create({ index: index_name, }); console.log("Creating index:"); console.log(response.body); // Add a document to the index. var document = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" }; var response = await client.index({ index: index_name, body: document }); console.log(response.body); } search().catch(console.log);

这一类似的示例使用 aws-opensearch-connector 而不是 aws4。您必须为 host 提供一个值。

const { Client } = require("@opensearch-project/opensearch"); const { defaultProvider } = require("@aws-sdk/credential-provider-node"); const createAwsOpensearchConnector = require("aws-opensearch-connector"); var host = '' // e.g. https://my-domain.region.es.amazonaws.com const getClient = async () => { const awsCredentials = await defaultProvider()(); const connector = createAwsOpensearchConnector({ credentials: awsCredentials, region: process.env.AWS_REGION ?? 'us-east-1', getCredentials: function(cb) { return cb(); } }); return new Client({ ...connector, node: host, }); } async function search() { // Initialize the client. var client = await getClient(); // Create an index. var index_name = "test-index"; var response = await client.indices.create({ index: index_name, }); console.log("Creating index:"); console.log(response.body); // Add a document to the index. var document = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" }; var response = await client.index({ index: index_name, body: document }); console.log(response.body); } search().catch(console.log);

如果您不想使用 opensearch-js,则可以仅发出标准 HTTP 请求。本节包含适用于 JavaScript in Node.js 的 SDK 的版本 2 和版本 3。尽管版本 2 作为单个包发布,但版本 3 具有模块化体系结构,每项服务都有单独的包。

Version 3

此示例使用适用于 JavaScript in Node.js 的 SDK 版本 3。从终端设备运行以下命令:

npm i @aws-sdk/protocol-http npm i @aws-sdk/credential-provider-node npm i @aws-sdk/signature-v4 npm i @aws-sdk/node-http-handler npm i @aws-crypto/sha256-browser

此示例代码将为单个文档编制索引。必须提供 regiondomain 的值。

const { HttpRequest} = require("@aws-sdk/protocol-http"); const { defaultProvider } = require("@aws-sdk/credential-provider-node"); const { SignatureV4 } = require("@aws-sdk/signature-v4"); const { NodeHttpHandler } = require("@aws-sdk/node-http-handler"); const { Sha256 } = require("@aws-crypto/sha256-browser"); var region = ''; // e.g. us-west-1 var domain = ''; // e.g. search-domain.region.es.amazonaws.com var index = 'node-test'; var type = '_doc'; var id = '1'; var json = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" }; indexDocument(json).then(() => process.exit()) async function indexDocument(document) { // Create the HTTP request var request = new HttpRequest({ body: JSON.stringify(document), headers: { 'Content-Type': 'application/json', 'host': domain }, hostname: domain, method: 'PUT', path: index + '/' + type + '/' + id }); // Sign the request var signer = new SignatureV4({ credentials: defaultProvider(), region: region, service: 'es', sha256: Sha256 }); var signedRequest = await signer.sign(request); // Send the request var client = new NodeHttpHandler(); var { response } = await client.handle(signedRequest) console.log(response.statusCode + ' ' + response.body.statusMessage); var responseBody = ''; await new Promise(() => { response.body.on('data', (chunk) => { responseBody += chunk; }); response.body.on('end', () => { console.log('Response body: ' + responseBody); }); }).catch((error) => { console.log('Error: ' + error); }); };
Version 2

此示例使用适用于 JavaScript in Node.js 的 SDK 版本 2。从终端运行以下命令:

npm install aws-sdk

此示例代码将为单个文档编制索引。必须提供 regiondomain 的值。

var AWS = require('aws-sdk'); var region = ''; // e.g. us-west-1 var domain = ''; // e.g. search-domain.region.es.amazonaws.com var index = 'node-test'; var type = '_doc'; var id = '1'; var json = { "title": "Moneyball", "director": "Bennett Miller", "year": "2011" } indexDocument(json); function indexDocument(document) { var endpoint = new AWS.Endpoint(domain); var request = new AWS.HttpRequest(endpoint, region); request.method = 'PUT'; request.path += index + '/' + type + '/' + id; request.body = JSON.stringify(document); request.headers['host'] = domain; request.headers['Content-Type'] = 'application/json'; request.headers['Content-Length'] = Buffer.byteLength(request.body); var credentials = new AWS.EnvironmentCredentials('AWS'); var signer = new AWS.Signers.V4(request, 'es'); signer.addAuthorization(credentials, new Date()); var client = new AWS.HttpClient(); return new Promise((resolve, reject) => { client.handleRequest( request, null, (response) => { const {statusCode, statusMessage, headers} = response; let body = ''; response.on('data', (chunk) => { body += chunk; }); response.on('end', () => { const data = {statusCode, statusMessage, headers}; if (body) { data.body = body; } resolve(data); console.log("Response body:" + body); }); }, (error) => { reject(error); console.log("Error:" + error) } ); }) };

如果凭证不起作用,则在终端使用以下命令将其导出:

export AWS_ACCESS_KEY_ID="your-access-key" export AWS_SECRET_ACCESS_KEY="your-secret-key" export AWS_SESSION_TOKEN="your-session-token"

Go

此示例使用 Amazon SDK for Go 并为单个文档编制索引。必须提供 domainregion 的值。

package main import ( "fmt" "net/http" "strings" "time" "github.com/aws/aws-sdk-go/aws/credentials" "github.com/aws/aws-sdk-go/aws/signer/v4" ) func main() { // Basic information for the Amazon OpenSearch Service domain domain := "" // e.g. https://my-domain.region.es.amazonaws.com index := "my-index" id := "1" endpoint := domain + "/" + index + "/" + "_doc" + "/" + id region := "" // e.g. us-east-1 service := "es" // Sample JSON document to be included as the request body json := `{ "title": "Thor: Ragnarok", "director": "Taika Waititi", "year": "2017" }` body := strings.NewReader(json) // Get credentials from environment variables and create the Signature Version 4 signer credentials := credentials.NewEnvCredentials() signer := v4.NewSigner(credentials) // An HTTP client for sending the request client := &http.Client{} // Form the HTTP request req, err := http.NewRequest(http.MethodPut, endpoint, body) if err != nil { fmt.Print(err) } // You can probably infer Content-Type programmatically, but here, we just say that it's JSON req.Header.Add("Content-Type", "application/json") // Sign the request, send it, and print the response signer.Sign(req, body, service, region, time.Now()) resp, err := client.Do(req) if err != nil { fmt.Print(err) } fmt.Print(resp.Status + "\n") }

如果凭证不起作用,则在终端使用以下命令将其导出:

export AWS_ACCESS_KEY_ID="your-access-key" export AWS_SECRET_ACCESS_KEY="your-secret-key" export AWS_SESSION_TOKEN="your-session-token"