UpdateStream

On This Page

Description

Updates a stream's configuration by increasing its shard count. The changes are applied immediately.

Spark-Streaming Note
In v3.6.1, to use the Spark Streaming API to consume records from new shards after a shard-count increase, you must first restart the consumer application.

Request

Request Header

Syntax
    POST /<container>/<resource> HTTP/1.1
Host: <web-APIs URL>
Content-Type: application/json
X-v3io-function: UpdateStream
<Authorization OR X-v3io-session-key>: <value>

    url = "http://<web-APIs URL>/<container>/<resource>"
headers = {
            "Content-Type": "application/json",
            "X-v3io-function": "UpdateStream",
            "<Authorization OR X-v3io-session-key>": "<value>"
          }
    URL Resource Parameters

    The path to the stream to be updated. You can optionally set the stream name in the request's StreamName JSON parameter instead of in the URL.

    Request Data

    Syntax
      {
    "StreamName": "string",
    "ShardCount": number,
}

      payload = {
            "StreamName": "string",
            "ShardCount": number
          }
      Parameters
      StreamName

      The name of the stream to be updated.

      • Type: String
      • Requirement: Required if not set in the request URL
      ShardCount

      The stream's new shard count (total number of shards in the stream).
      The new shard count cannot be smaller than the current shard count.

      Note
      If you increase a stream's shard count after its creation, new records with a previously used partition key might be assigned to a new shard. See Stream Sharding and Partitioning, and PutRecords.
      • Type: Number
      • Requirement: Required

      Response

      Response Data

      None

      Errors

      In the event of an error, the response includes a JSON object with an ErrorCode element that contains a unique numeric error code, and an ErrorMessage element that contains one of the following API error messages:
      Error Message Description
      InvalidArgumentException A provided request parameter is not valid for this request.
      InvalidShardCountException The provided shard count is not larger than the current number of shards within the stream.
      ResourceNotFoundException The specified resource does not exist.
      ResourceNotStreamException The specified resource is not a stream.
      Permission denied The sender of the request does not have the required permissions to perform the operation.

      Examples

      Update the shard count of a MyStream stream to 200:

      Request
        POST /mycontainer/MyStream/ HTTP/1.1
Host: https://default-tenant.app.mycluster.iguazio.com:8443
Content-Type: application/json
X-v3io-function: UpdateStream
X-v3io-session-key: e8bd4ca2-537b-4175-bf01-8c74963e90bf

        {
    "ShardCount": 200
}

        import requests

url = "https://default-tenant.app.mycluster.iguazio.com:8443/mycontainer/MyStream/"
headers = {
            "Content-Type": "application/json",
            "X-v3io-function": "UpdateStream",
            "X-v3io-session-key": "e8bd4ca2-537b-4175-bf01-8c74963e90bf"
          }
payload = {"ShardCount": 200}

response = requests.post(url, json=payload, headers=headers)
print(response.text)
        Response
        HTTP/1.1 200 OK
Content-Type: application/json