Athena

Kubemq athena target connector allows services using kubemq server to access aws athena service.

Prerequisites

The following required to run the aws-athena target connector:

• kubemq cluster

• aws account with athena active service

• kubemq-source deployment

Configuration

athena target connector configuration properties:

Properties Key	Required	Description	Example
aws_key	yes	aws key	aws key supplied by aws
aws_secret_key	yes	aws secret key	aws secret key supplied by aws
region	yes	region	aws region
token	no	aws token ("default" empty string	aws token

Example:

bindings:
  - name: kubemq-query-aws-athena
    source:
      kind: kubemq.query
      name: kubemq-query
      properties:
        address: "kubemq-cluster:50000"
        client_id: "kubemq-query-aws-athena"
        auth_token: ""
        channel: "query.aws.athena"
        group:   ""
        auto_reconnect: "true"
        reconnect_interval_seconds: "1"
        max_reconnects: "0"
    target:
      kind: aws.athena
      name: aws-athena
      properties:
        aws_key: "id"
        aws_secret_key: 'json'
        region:  "region"
        token: ""

Usage

List Catalogs

list all catalogs

List Catalogs:

Metadata Key	Required	Description	Possible values
method	yes	type of method	"list_data_catalogs"

Example:

{
  "metadata": {
    "method": "list_data_catalogs"
  },
  "data": null
}

List Databases

list all databases

List Databases:

Metadata Key	Required	Description	Possible values
method	yes	type of method	"list_databases"
catalog	yes	aws athena catalog	"string"

Example:

{
  "metadata": {
    "method": "list_databases",
    "catalog": "my_catalog"
  },
  "data": null
}

Query

create a query request return execution_id.

Query:

Metadata Key	Required	Description	Possible values
method	yes	type of method	"query"
catalog	yes	aws athena catalog	"string"
db	yes	aws athena db name	"string"
output_location	yes	aws athena folder location	"string"
query	yes	aws query to execute	"query"

Example:

{
  "metadata": {
    "method": "query",
    "catalog": "my_catalog",
    "db": "my_db",
    "output_location": "my_output_location/path",
    "query": "SELECT * FROM \"my_db\".\"new_table_name2\" limit 10;"
  },
  "data": null
}

Get Query Result

get Query result by execution_id that return from Query result.

Get Query Result:

Metadata Key	Required	Description	Possible values
method	yes	type of method	"get_query_result"
execution_id	yes	aws executionID that returned from query	"string"

Example:

{
  "metadata": {
    "method": "get_query_result",
    "execution_id": "123-456-789"
  },
  "data": null
}