Athena
Kubemq athena target connector allows services using kubemq server to access aws athena service.
Prerequisites
The following required to run the aws-athena target connector:
kubemq cluster
aws account with athena active service
kubemq-source deployment
Configuration
athena target connector configuration properties:
Properties Key
Required
Description
Example
aws_key
yes
aws key
aws key supplied by aws
aws_secret_key
yes
aws secret key
aws secret key supplied by aws
region
yes
region
aws region
token
no
aws token ("default" empty string
aws token
Example:
bindings:
- name: kubemq-query-aws-athena
source:
kind: kubemq.query
name: kubemq-query
properties:
address: "kubemq-cluster:50000"
client_id: "kubemq-query-aws-athena"
auth_token: ""
channel: "query.aws.athena"
group: ""
auto_reconnect: "true"
reconnect_interval_seconds: "1"
max_reconnects: "0"
target:
kind: aws.athena
name: aws-athena
properties:
aws_key: "id"
aws_secret_key: 'json'
region: "region"
token: ""
Usage
List Catalogs
list all catalogs
List Catalogs:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"list_data_catalogs"
Example:
{
"metadata": {
"method": "list_data_catalogs"
},
"data": null
}
List Databases
list all databases
List Databases:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"list_databases"
catalog
yes
aws athena catalog
"string"
Example:
{
"metadata": {
"method": "list_databases",
"catalog": "my_catalog"
},
"data": null
}
Query
create a query request return execution_id.
Query:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"query"
catalog
yes
aws athena catalog
"string"
db
yes
aws athena db name
"string"
output_location
yes
aws athena folder location
"string"
query
yes
aws query to execute
"query"
Example:
{
"metadata": {
"method": "query",
"catalog": "my_catalog",
"db": "my_db",
"output_location": "my_output_location/path",
"query": "SELECT * FROM \"my_db\".\"new_table_name2\" limit 10;"
},
"data": null
}
Get Query Result
get Query result by execution_id that return from Query result.
Get Query Result:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"get_query_result"
execution_id
yes
aws executionID that returned from query
"string"
Example:
{
"metadata": {
"method": "get_query_result",
"execution_id": "123-456-789"
},
"data": null
}
Last updated
Was this helpful?