HireHire › TCS jobs › Kafka

Kafka

Apply for this role or explore on the map →

1.     TIBCO EMS to Kafka Migration

  • Assess the existing TIBCO EMS infrastructure (topics, queues, message flows, and integrations) and design a migration strategy to Kafka.
  • Map TIBCO EMS constructs (e.g., topics, bridges, routing) to Kafka equivalents (e.g., Kafka topics, partitions, consumers, Kafka Connect).
  • Implement proof-of-concepts (PoCs) to validate performance, latency, and compatibility.
  • Develop custom connectors/adapters (if needed) to ensure backward compatibility with legacy systems.

 

      2.     Kafka Architecture & Development

  • Design, configure, and optimize Kafka clusters (on-premises/cloud) for high throughput, fault tolerance, and scalability.
  • Develop producers/consumers in Java/Spring Boot (or Python/Scala) with efficient serialization (Avro/Protobuf/JSON).
  • Implement Kafka Streams or ksqlDB for real-time stream processing (e.g., aggregations, joins, filtering).
  • Set up schema management (Confluent Schema Registry or Apicurio) and enforce governance policies.

 

      3.     Performance & Reliability

  • Monitor and tune Kafka performance (e.g., broker configuration, partition sizing, replication factors).
  • Implement exactly-once semantics, idempotent producers, and transa ctional messaging where required.
  • Design disaster recovery (DR) and backup strategies for Kafka clusters.
  • Conduct load testing and benchmarking to ensure SLAs are met.

 

       4.    Integration & DevOps

  • Collaborate with DevOps/SRE teams to deploy Kafka in containerized environments (Kubernetes/Docker) using IaC (Terraform/Ansible).
  • Integrate Kafka with CI/CD pipelines (Jenkins, GitLab CI, ArgoCD) for automated testing and deployment.
  • Ensure security best practices: TLS encryption, SASL/SCRAM authentication, RBAC (via Kafka ACLs or Confluent RBAC).

 

       5.    Documentation & Knowledge Sharing