Athena
Kubemq athena target connector allows services using kubemq server to access aws athena service.

Prerequisites

The following required to run the aws-athena target connector:
  • kubemq cluster
  • aws account with athena active service
  • kubemq-source deployment

Configuration

athena target connector configuration properties:
Properties Key
Required
Description
Example
aws_key
yes
aws key
aws key supplied by aws
aws_secret_key
yes
aws secret key
aws secret key supplied by aws
region
yes
region
aws region
token
no
aws token ("default" empty string
aws token
Example:
1
bindings:
2
- name: kubemq-query-aws-athena
3
source:
4
kind: kubemq.query
5
name: kubemq-query
6
properties:
7
address: "kubemq-cluster:50000"
8
client_id: "kubemq-query-aws-athena"
9
auth_token: ""
10
channel: "query.aws.athena"
11
group: ""
12
auto_reconnect: "true"
13
reconnect_interval_seconds: "1"
14
max_reconnects: "0"
15
target:
16
kind: aws.athena
17
name: aws-athena
18
properties:
19
aws_key: "id"
20
aws_secret_key: 'json'
21
region: "region"
22
token: ""
Copied!

Usage

List Catalogs

list all catalogs
List Catalogs:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"list_data_catalogs"
Example:
1
{
2
"metadata": {
3
"method": "list_data_catalogs"
4
},
5
"data": null
6
}
Copied!

List Databases

list all databases
List Databases:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"list_databases"
catalog
yes
aws athena catalog
"string"
Example:
1
{
2
"metadata": {
3
"method": "list_databases",
4
"catalog": "my_catalog"
5
},
6
"data": null
7
}
Copied!

Query

create a query request return execution_id.
Query:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"query"
catalog
yes
aws athena catalog
"string"
db
yes
aws athena db name
"string"
output_location
yes
aws athena folder location
"string"
query
yes
aws query to execute
"query"
Example:
1
{
2
"metadata": {
3
"method": "query",
4
"catalog": "my_catalog",
5
"db": "my_db",
6
"output_location": "my_output_location/path",
7
"query": "SELECT * FROM \"my_db\".\"new_table_name2\" limit 10;"
8
},
9
"data": null
10
}
Copied!

Get Query Result

get Query result by execution_id that return from Query result.
Get Query Result:
Metadata Key
Required
Description
Possible values
method
yes
type of method
"get_query_result"
execution_id
yes
aws executionID that returned from query
"string"
Example:
1
{
2
"metadata": {
3
"method": "get_query_result",
4
"execution_id": "123-456-789"
5
},
6
"data": null
7
}
Copied!
Last modified 11mo ago