# Kinesis

Kubemq kinesis target connector allows services using kubemq server to access aws kinesis service.

## Prerequisites

The following required to run the aws-kinesis target connector:

* kubemq cluster
* aws account with kinesis active service
* kubemq-source deployment

## Configuration

sns target connector configuration properties:

| Properties Key   | Required | Description                       | Example                        |
| ---------------- | -------- | --------------------------------- | ------------------------------ |
| aws\_key         | yes      | aws key                           | aws key supplied by aws        |
| aws\_secret\_key | yes      | aws secret key                    | aws secret key supplied by aws |
| region           | yes      | region                            | aws region                     |
| token            | no       | aws token ("default" empty string | aws token                      |

Example:

```yaml
bindings:
  - name: kubemq-query-aws-kinesis
    source:
      kind: kubemq.query
      name: kubemq-query
      properties:
        address: "kubemq-cluster:50000"
        client_id: "kubemq-query-aws-kinesis"
        auth_token: ""
        channel: "query.aws.kinesis"
        group:   ""
        auto_reconnect: "true"
        reconnect_interval_seconds: "1"
        max_reconnects: "0"
    target:
      kind: aws.kinesis
      name: aws-kinesis
      properties:
        aws_key: "id"
        aws_secret_key: 'json'
        region:  "region"
        token: ""
```

## Usage

### List Streams

list kinesis streams

List Streams:

| Metadata Key | Required | Description    | Possible values |
| ------------ | -------- | -------------- | --------------- |
| method       | yes      | type of method | "list\_streams" |

Example:

```javascript
{
  "metadata": {
    "method": "list_streams"
  },
  "data": null
}
```

### List Stream Consumers

list kinesis Stream Consumers.

List Stream Consumers:

| Metadata Key | Required | Description                          | Possible values           |
| ------------ | -------- | ------------------------------------ | ------------------------- |
| method       | yes      | type of method                       | "list\_stream\_consumers" |
| stream\_arn  | yes      | aws stream arn of the desired stream | "arn::mystream"           |

Example:

```javascript
{
  "metadata": {
    "method": "list_stream_consumers",
    "stream_arn": "arn::mystream"
  },
  "data": null
}
```

### Create Stream

Create a kinesis Stream.

Create Stream:

| Metadata Key | Required | Description                            | Possible values  |
| ------------ | -------- | -------------------------------------- | ---------------- |
| method       | yes      | type of method                         | "create\_stream" |
| stream\_name | yes      | aws stream name as string              | "string"         |
| shard\_count | no       | number of shards to create (default 1) | "1"              |

Example:

```javascript
{
  "metadata": {
    "method": "create_stream",
    "stream_name": "my_stream",
    "shard_count": "1"
  },
  "data": null
}
```

### List Shards

list stream Shards .

List Shards:

| Metadata Key | Required | Description               | Possible values |
| ------------ | -------- | ------------------------- | --------------- |
| method       | yes      | type of method            | "list\_shards"  |
| stream\_name | yes      | aws stream name as string | "string"        |

Example:

```javascript
{
  "metadata": {
    "method": "list_shards",
    "stream_name": "my_stream"
  },
  "data": null
}
```

### Get Shard Iterator

Get Shard Iterator used to preform get data .

Get Shard Iterator:

| Metadata Key          | Required | Description                                          | Possible values                                                                         |
| --------------------- | -------- | ---------------------------------------------------- | --------------------------------------------------------------------------------------- |
| method                | yes      | type of method                                       | "get\_shard\_iterator"                                                                  |
| stream\_name          | yes      | aws stream name as string                            | "string"                                                                                |
| shard\_iterator\_type | yes      | aws shard iterator type                              | see <https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html> |
| shard\_id             | yes      | aws shard full id (can be acquired using list shard) | see <https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html> |

Example:

```javascript
{
  "metadata": {
    "method": "get_shard_iterator",
    "shard_iterator_type": "LATEST",
    "stream_name": "my_stream",
    "shard_id": "8619-AWE1"
  },
  "data": null
}
```

### Put Record

Send data to stream .

Put Record:

| Metadata Key   | Required | Description                                                          | Possible values                                                                  |
| -------------- | -------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| method         | yes      | type of method                                                       | "put\_record"                                                                    |
| stream\_name   | yes      | aws stream name as string                                            | "string"                                                                         |
| partition\_key | yes      | determines which shard in the stream the data record is assigned to. | see <https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html> |
| data           | yes      | Message to send to stream                                            | string                                                                           |

Example:

```javascript
{
  "metadata": {
    "method": "put_record",
    "partition_key": "0356",
    "stream_name": "my_stream"
  },
  "data": "eyJteV9yZXN1bHQiOiJvayJ9"
}
```

### Put Records

Send multi data to a stream .

Put Records:

| Metadata Key | Required | Description                                                   | Possible values                                                                                                                        |
| ------------ | -------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| method       | yes      | type of method                                                | "put\_records"                                                                                                                         |
| stream\_name | yes      | aws stream name as string                                     | "string"                                                                                                                               |
| data         | yes      | Key value pair of partition\_key(string) and message(\[]byte) | key value of partition\_key and message as key value pair <https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html> |

Example:

```javascript
{
  "metadata": {
    "method": "put_records",
    "stream_name": "my_stream"
  },
  "data": "eyIxIjoiZXlKdGVWOXlaWE4xYkhRaU9pSnZheUo5IiwiMiI6ImV5SnRlVjl5WlhOMWJIUXlJam9pYjJzaEluMD0ifQ=="
}
```

### Put Records

Send multi data to a stream .

Put Records:

| Metadata Key | Required | Description                                                   | Possible values                                                                                                                        |
| ------------ | -------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| method       | yes      | type of method                                                | "put\_records"                                                                                                                         |
| stream\_name | yes      | aws stream name as string                                     | "string"                                                                                                                               |
| data         | yes      | Key value pair of partition\_key(string) and message(\[]byte) | key value of partition\_key and message as key value pair <https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html> |

Example:

```javascript
{
  "metadata": {
    "method": "put_records",
    "stream_name": "my_stream"
  },
  "data": "eyIxIjoiZXlKdGVWOXlaWE4xYkhRaU9pSnZheUo5IiwiMiI6ImV5SnRlVjl5WlhOMWJIUXlJam9pYjJzaEluMD0ifQ=="
}
```

### Get Records

Get multi data from a stream .

Get Records:

| Metadata Key | Required | Description                                  | Possible values |
| ------------ | -------- | -------------------------------------------- | --------------- |
| method       | yes      | type of method                               | "get\_records"  |
| stream\_name | yes      | aws stream name as string                    | "string"        |
| limit        | no       | Number of limit message to get (default "1") | "int value"     |

Example:

```javascript
{
  "metadata": {
    "method": "put_records",
    "stream_name": "my_stream",
    "limit": "1"
  },
  "data": null
}
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.kubemq.io/configuration/connectors/kubemq-targets/aws/kinesis.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.