Hello Alexey,

There are many ways in which you can solve this problem. In this case I took it as a simple example to explain the use of threads in PySpark.

The code will not run on driver rather on executors, as this a spark code we are processing on individual threads. Driver will only be used to schedule the tasks.