Kafka / Docker Tips

This page just contains some useful Kafka and Docker information I refer to from time to time.

Kafka Tips

How to get a list of Kafka topics?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --list

How to get info on a Kafka topic?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic core-eventingest

How to create a Kafka topic?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic core-events --partitions 3 --replication-factor 1

How do I get a list of the Kafka bin folder?

docker exec -it kafka ls /opt/kafka_2.13-2.8.0/bin/

How do I consume a topic from the console?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --formatter kafka.tools.DefaultMessageFormatter --property print.timestamp=true --property print.key=true --property print.value=true

Docker

Command	Purpose
docker-compose up|down	brings docker images up or down as per the yaml file
docker-compose rm -svf	removes stopped service containers
docker images	displays a list of docker images
docker ps	shows docker images currently running
docker container ls –all	lists all docker containers
docker volume ls	lists all docker volumes
docker container rm id	removes the container with the specified id
docker volume rm id	removes the volume with the specified id
docker stop id	shutdown the container with the specified id
docker kill $(docker ps -q)	shutdown all running containers
docker ps -q | xargs docker stop	another way to shutdown all running containers
docker exec -ti id cmd	executes the specified command in the container with the specified id. For example: docker exec -ti 7717da13fcbc sh -c "echo a && echo b"

Basic ASP.NET 6 DockerFile

The following examples are modified from Les Jackson’s excellent free course, see here.

Add Dockerfile to your project, in the root folder:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /app

COPY *.csproj ./
RUN dotnet restore

COPY . ./
RUN dotnet publish -c Release -o out

FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "<service name>.dll"]

Docker Commands for a Basic ASP.NET 6 Application

Execute the commands from a terminal, from the root folder of the project, where the .csproj file is located. Docker Desktop should be running. You can create a docker hub id here.

docker build -t <docker hub id>/<service name> .

docker run -p 8080:80 -d <docker hub id>/<service name>

docker ps

docker stop <container id>

docker start <container id>

docker login

docker push <docker hub id>/<service name>

Example Kubernetes Deploy file for the Basic Application

From the same course, a sample Kubernetes deploy file named platforms-depl.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: platform-depl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: platformservice
  template:
    metadata:
      labels:
        app: platformservice
    spec:
      containers:
        - name: platformservice
          image: beyondvelocity/platformservice:latest

Example Kubernates NodePort file for the Basic Application

From the same course, a sample Kubernetes NodePort creation file named platforms-np-srv.yaml:

apiVersion: v1
kind: Service
metadata:
  name: platformnpservice-srv
spec:
  type: NodePort
  selector:
    app: platformservice
  ports:
    - name: platformservice
      protocol: TCP
      port: 80
      targetPort: 80

KubeCtl Commands for the Basic Application

With Kubernetes enabled within Docker Desktop, use the following comands:

kubectl version

kubectl apply -f platforms-depl.yaml

kubectl get deployments

kubectl get pods

kubectl delete deployment <name>

kubectl apply -f platforms-np-srv.yaml

kubectl get services

Installing Docker Compose in Ubuntu

There’s plenty of online help for installing Docker, but for Docker-Compose 2.0, help was harder to find, these links were useful:

Install Docker in Ubuntu 21.10, Install Dockercompose on Ubuntu 21.10

How to Install the Lastest Docker Compose on Linux

Host Configuration

Add the following hosts to /etc/hosts:

127.0.0.1 kafka
127.0.0.1 kubernetes.docker.internal
172.17.0.1 host.docker.internal

Advertisement