У меня есть система, файлы которой загружаются на SQL Server через приложение. Моя цель - передать эти загруженные файлы в MS SQL Server в Minio через Kafka.
Я установил брукеры Kafka через Strimzi. Затем я подготовил собственный образ Docker, который включает в себя Kafka Connect, плагин Debezium SQL и плагин S3 Sink Connector.
Это мой Dockerfile:
FROM strimzi/kafka:latest-kafka-2.5.0
USER root:root
COPY ./plugins/ /opt/kafka/plugins/
USER 1001
ENV MINIO_VOLUMES=http://minio.dev-kik.io
ENV AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
ENV AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
... затем я развернул свой контейнер через Strimzi и установил кластер Kafka Connect на Kubernetes. Я установил sql-коннектор, он работает.
Когда я запускаю коннектор S3/Minio, возникает несколько проблем, которые я не могу полностью интерпретировать. 1) Я не уверен, что ввод учетных данных Minio в Dockerfile в качестве переменной env верен или нет. 2) Хотя модуль (ы) Kafka Connect может разрешать URL-адрес minio в командной строке bash, Kafka connect говорит, что мой URL-адрес является неизвестным адресом. 3) Я не уверен, что ввожу свой адрес Minio в правильный компонент.
моя конфигурация MinioConnector (Strimzi):
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: "minio-connector"
labels:
strimzi.io/cluster: mssql-minio-connect-cluster
spec:
class: io.confluent.connect.s3.S3SinkConnector
config:
storage.class: io.confluent.connect.s3.storage.S3Storage
format.class: io.confluent.connect.s3.format.avro.AvroFormat
schema.generator.class: io.confluent.connect.storage.hive.schema.DefaultSchemaGenerator
partitioner.class: io.confluent.connect.storage.partitioner.DefaultPartitioner
tasks.max: '1'
topics: filesql1.dbo.files
s3.bucket.name: dosyalar
s3.part.size: '5242880'
flush.size: '3'
format: binary
schema.compatibility: NONE
max.request.size: "536870912"
store.url: http://minio.dev-kik.io
...и журналы:
[StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,050 INFO S3SinkConnectorConfig values:
avro.codec = null
aws.access.key.id =
aws.secret.access.key = [hidden]
behavior.on.null.values = fail
connect.meta.data = true
enhanced.avro.schema.support = false
filename.offset.zero.pad.width = 10
flush.size = 3
format.bytearray.extension = .bin
format.bytearray.separator = null
format.class = class io.confluent.connect.s3.format.avro.AvroFormat
parquet.codec = snappy
retry.backoff.ms = 5000
rotate.interval.ms = -1
rotate.schedule.interval.ms = -1
s3.acl.canned = null
s3.bucket.name = dosyalar
s3.compression.level = -1
s3.compression.type = none
s3.credentials.provider.class = class com.amazonaws.auth.DefaultAWSCredentialsProviderChain
s3.http.send.expect.continue = true
s3.object.tagging = false
s3.part.retries = 3
s3.part.size = 5242880
s3.proxy.password = [hidden]
s3.proxy.url =
s3.proxy.user = null
s3.region = us-west-2
s3.retry.backoff.ms = 200
s3.sse.customer.key = [hidden]
s3.sse.kms.key.id =
s3.ssea.name =
s3.wan.mode = false
schema.cache.size = 1000
schema.compatibility = NONE
shutdown.timeout.ms = 3000
(io.confluent.connect.s3.S3SinkConnectorConfig) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,051 INFO StorageCommonConfig values:
directory.delim = /
file.delim = +
storage.class = class io.confluent.connect.s3.storage.S3Storage
store.url = http://minio.dev-kik.io
topics.dir = topics
(io.confluent.connect.storage.common.StorageCommonConfig) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,052 INFO PartitionerConfig values:
locale =
partition.duration.ms = -1
partition.field.name = []
partitioner.class = class io.confluent.connect.storage.partitioner.DefaultPartitioner
path.format =
timestamp.extractor = Wallclock
timestamp.field = timestamp
timezone =
(io.confluent.connect.storage.partitioner.PartitionerConfig) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,054 INFO Creating task minio-connector-0 (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,054 INFO Starting S3 connector minio-connector (io.confluent.connect.s3.S3SinkConnector) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,064 INFO Finished creating connector minio-connector (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,074 INFO ConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.connect.s3.S3SinkConnector
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = minio-connector
tasks.max = 1
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,074 INFO EnrichedConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.connect.s3.S3SinkConnector
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = minio-connector
tasks.max = 1
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,079 INFO SinkConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.connect.s3.S3SinkConnector
errors.deadletterqueue.context.headers.enable = false
errors.deadletterqueue.topic.name =
errors.deadletterqueue.topic.replication.factor = 3
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = minio-connector
tasks.max = 1
topics = [filesql1.dbo.files]
topics.regex =
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.SinkConnectorConfig) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,079 INFO EnrichedConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.connect.s3.S3SinkConnector
errors.deadletterqueue.context.headers.enable = false
errors.deadletterqueue.topic.name =
errors.deadletterqueue.topic.replication.factor = 3
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = minio-connector
tasks.max = 1
topics = [filesql1.dbo.files]
topics.regex =
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig) [StartAndStopExecutor-connect-1-3]
2020-05-17 07:48:00,121 INFO TaskConfig values:
task.class = class io.confluent.connect.s3.S3SinkTask
(org.apache.kafka.connect.runtime.TaskConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,124 INFO Instantiated task minio-connector-0 with version 5.5.0 of type io.confluent.connect.s3.S3SinkTask (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,125 INFO JsonConverterConfig values:
converter.type = key
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
(org.apache.kafka.connect.json.JsonConverterConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,125 INFO Set up the key converter class org.apache.kafka.connect.json.JsonConverter for task minio-connector-0 using the worker config (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,126 INFO JsonConverterConfig values:
converter.type = value
decimal.format = BASE64
schemas.cache.size = 1000
schemas.enable = true
(org.apache.kafka.connect.json.JsonConverterConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,126 INFO Set up the value converter class org.apache.kafka.connect.json.JsonConverter for task minio-connector-0 using the worker config (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,127 INFO Set up the header converter class org.apache.kafka.connect.storage.SimpleHeaderConverter for task minio-connector-0 using the worker config (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,130 INFO Initializing: org.apache.kafka.connect.runtime.TransformationChain{} (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,131 INFO SinkConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.connect.s3.S3SinkConnector
errors.deadletterqueue.context.headers.enable = false
errors.deadletterqueue.topic.name =
errors.deadletterqueue.topic.replication.factor = 3
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = minio-connector
tasks.max = 1
topics = [filesql1.dbo.files]
topics.regex =
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.SinkConnectorConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,132 INFO EnrichedConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.connect.s3.S3SinkConnector
errors.deadletterqueue.context.headers.enable = false
errors.deadletterqueue.topic.name =
errors.deadletterqueue.topic.replication.factor = 3
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = null
name = minio-connector
tasks.max = 1
topics = [filesql1.dbo.files]
topics.regex =
transforms = []
value.converter = null
(org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,135 INFO ConsumerConfig values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [kafka-beytepe-kafka-bootstrap:9093]
check.crcs = true
client.dns.lookup = default
client.id = connector-consumer-minio-connector-0
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = connect-minio-connector
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = SSL
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.2
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = /tmp/kafka/cluster.truststore.p12
ssl.truststore.password = [hidden]
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
(org.apache.kafka.clients.consumer.ConsumerConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,425 WARN The configuration 'offset.flush.timeout.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,425 WARN The configuration 'max.request.size' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,425 INFO Kafka version: 2.5.0 (org.apache.kafka.common.utils.AppInfoParser) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,425 INFO Kafka commitId: 66563e712b0b9f84 (org.apache.kafka.common.utils.AppInfoParser) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,425 INFO Kafka startTimeMs: 1589701680425 (org.apache.kafka.common.utils.AppInfoParser) [StartAndStopExecutor-connect-1-4]
2020-05-17 07:48:00,445 INFO [Worker clientId=connect-1, groupId=sql-minio-cluster] Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [DistributedHerder-connect-1-1]
2020-05-17 07:48:00,465 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Subscribed to topic(s): filesql1.dbo.files (org.apache.kafka.clients.consumer.KafkaConsumer) [task-thread-minio-connector-0]
2020-05-17 07:48:00,470 INFO S3SinkConnectorConfig values:
avro.codec = null
aws.access.key.id =
aws.secret.access.key = [hidden]
behavior.on.null.values = fail
connect.meta.data = true
enhanced.avro.schema.support = false
filename.offset.zero.pad.width = 10
flush.size = 3
format.bytearray.extension = .bin
format.bytearray.separator = null
format.class = class io.confluent.connect.s3.format.avro.AvroFormat
parquet.codec = snappy
retry.backoff.ms = 5000
rotate.interval.ms = -1
rotate.schedule.interval.ms = -1
s3.acl.canned = null
s3.bucket.name = dosyalar
s3.compression.level = -1
s3.compression.type = none
s3.credentials.provider.class = class com.amazonaws.auth.DefaultAWSCredentialsProviderChain
s3.http.send.expect.continue = true
s3.object.tagging = false
s3.part.retries = 3
s3.part.size = 5242880
s3.proxy.password = [hidden]
s3.proxy.url =
s3.proxy.user = null
s3.region = us-west-2
s3.retry.backoff.ms = 200
s3.sse.customer.key = [hidden]
s3.sse.kms.key.id =
s3.ssea.name =
s3.wan.mode = false
schema.cache.size = 1000
schema.compatibility = NONE
shutdown.timeout.ms = 3000
(io.confluent.connect.s3.S3SinkConnectorConfig) [task-thread-minio-connector-0]
2020-05-17 07:48:00,474 INFO StorageCommonConfig values:
directory.delim = /
file.delim = +
storage.class = class io.confluent.connect.s3.storage.S3Storage
store.url = http://minio.dev-kik.io
topics.dir = topics
(io.confluent.connect.storage.common.StorageCommonConfig) [task-thread-minio-connector-0]
2020-05-17 07:48:00,483 INFO PartitionerConfig values:
locale =
partition.duration.ms = -1
partition.field.name = []
partitioner.class = class io.confluent.connect.storage.partitioner.DefaultPartitioner
path.format =
timestamp.extractor = Wallclock
timestamp.field = timestamp
timezone =
(io.confluent.connect.storage.partitioner.PartitionerConfig) [task-thread-minio-connector-0]
2020-05-17 07:48:00,881 INFO Returning new credentials provider based on the configured credentials provider class (io.confluent.connect.s3.storage.S3Storage) [task-thread-minio-connector-0]
2020-05-17 07:48:02,007 INFO AvroDataConfig values:
connect.meta.data = true
enhanced.avro.schema.support = false
schemas.cache.config = 1000
(io.confluent.connect.avro.AvroDataConfig) [task-thread-minio-connector-0]
2020-05-17 07:48:02,010 INFO Started S3 connector task with assigned partitions: [] (io.confluent.connect.s3.S3SinkTask) [task-thread-minio-connector-0]
2020-05-17 07:48:02,011 INFO WorkerSinkTask{id=minio-connector-0} Sink task finished initialization and start (org.apache.kafka.connect.runtime.WorkerSinkTask) [task-thread-minio-connector-0]
2020-05-17 07:48:03,684 WARN [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Error while fetching metadata with correlation id 2 : {filesql1.dbo.files=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [task-thread-minio-connector-0]
2020-05-17 07:48:03,685 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Cluster ID: jnExZIqQT0y3UUvA-wSrQg (org.apache.kafka.clients.Metadata) [task-thread-minio-connector-0]
2020-05-17 07:48:03,687 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Discovered group coordinator kafka-beytepe-kafka-2.kafka-beytepe-kafka-brokers.kafka.svc:9093 (id: 2147483645 rack: null) (org.apache.kafka.clients.consumer.internals.AbstractCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:03,691 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:03,744 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Join group failed with org.apache.kafka.common.errors.MemberIdRequiredException: The group member needs to have a valid member id before actually entering a consumer group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:03,744 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:04,062 WARN [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Error while fetching metadata with correlation id 7 : {filesql1.dbo.files=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [task-thread-minio-connector-0]
2020-05-17 07:48:04,204 WARN [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Error while fetching metadata with correlation id 8 : {filesql1.dbo.files=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [task-thread-minio-connector-0]
2020-05-17 07:48:06,756 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Finished assignment for group at generation 1: {connector-consumer-minio-connector-0-5ebe1cc4-338c-46be-b636-2e8e1029a34c=Assignment(partitions=[filesql1.dbo.files-0])} (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:06,764 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Successfully joined group with generation 1 (org.apache.kafka.clients.consumer.internals.AbstractCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:06,765 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Adding newly assigned partitions: filesql1.dbo.files-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:06,774 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Found no committed offset for partition filesql1.dbo.files-0 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [task-thread-minio-connector-0]
2020-05-17 07:48:06,806 INFO [Consumer clientId=connector-consumer-minio-connector-0, groupId=connect-minio-connector] Resetting offset for partition filesql1.dbo.files-0 to offset 0. (org.apache.kafka.clients.consumer.internals.SubscriptionState) [task-thread-minio-connector-0]
2020-05-17 07:48:41,864 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:48:41,867 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:49:41,868 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:49:41,868 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:50:41,868 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:50:41,869 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:51:41,869 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:51:41,870 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:52:41,870 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:52:41,871 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:53:41,871 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:53:41,872 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:54:41,872 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:54:41,872 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:55:41,873 INFO WorkerSourceTask{id=mssql-files-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
2020-05-17 07:55:41,873 INFO WorkerSourceTask{id=mssql-files-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [SourceTaskOffsetCommitter-1]
Обратите внимание, что у меня их нет на AWS.
