Goroutine

This blog doesnt covers what is go or why goroutines are used in programs, what is the need etc, this covers some inner workings of goroutines

You would have used a Goroutine in ur Go programming language regularly even not known by you, in order to achieve concurrency or parallelism, these are nothing but lightweight struct with some metadata maintained by the goruntime and managed by the runtime in order to run our tasks concurrently, sometimes parallely too.

What are OS Threads#

OS Threads are fully managed by our kernel, not on application or not on the user space, our application(can be the complier or the direct application code) can create an os thread using a syscall pthread_create, this API accepts a function pointer that needs to be running on the new thread, upon passing the same, something like pthread_create(..,.., *fun_pointer,..) this fun_pointer function will starts executing.

Each individual OS threads are costly in terms of memory, also you are making your kernel to work more and so busy in terms of context switching, scheduling etc.

Consider the below C code, its a simple thread code, where am spawning 1M threads to do some cpu_heavy_lifting_work(a dummy), am creating 1M OS threads and trying to run cpu_heavy_lifting_work function in all these. you can try to run this code on ur machine.

Compile and run this code, in the console u will see Before creating threads, now calcualte the memory used and number of threads by this program using

memory usage: ps -o rss -p <PID>(printed in the program) thread_count: ps -M <PID> | wc -l

now after 20 seconds u will see After creating threads, now again calculate the memory used and number of threads by the above same command

#include <stdio.h>
#include <pthread.h>
#include <unistd.h>

void *cpu_heavy_lifting_work(void *arg) {
    // consider u may do some heavy lifting cpu task here
    sleep(120);
    return NULL;
}

int main() {
    int total_threads = 1000000;
    pthread_t threads[total_threads]; // just declares number of thread pointer on your application(user stack) that you will be passing
                                      // to kernel.

    printf("pid: %d\n", getpid());

    printf("Before creating threads\n");
    sleep(20);

    for (int i = 0; i < total_threads; i++) {
        // pass the thread pointers in here, pthread_create will do all the 
        // kernel level work like mmaping memory stack, kernel stack and not sure what more
        pthread_create(&threads[i], NULL, cpu_heavy_lifting_work, NULL);
    }
    printf("After creating threads\n");

    for (int i = 0; i < total_threads; i++) {
        // this is just a blocking call in main, to not kill this running code process.
        // think of it as wg.Wait in go terms
        pthread_join(threads[i], NULL);
    }

    return 0;
}

Below is a Go equivalent code to c, it does the same, but now calculate the same memory and os threads count, and u may see the difference.

func main() {
	var wg sync.WaitGroup
	total_threads := 10_00_000

	fmt.Println("pid: ", os.Getpid())
	fmt.Println("Before Creating Goroutines")

	time.Sleep(30 * time.Second)
	wg.Add(total_threads)

	for i := 0; i < total_threads; i++ {
		go func() {
			defer wg.Done()
			time.Sleep(120 * time.Second)
		}()
	}
	fmt.Println("After Creating Goroutines")

	wg.Wait()
}

From running the above two codes, you may really see the difference in OS threads count, not on the memory, even with goroutines go may take more memory, but the only difference is in c, its creating 1M os threads and handling all the context switching, scheduling, prioritizing tasks etc are all handled now to kernel, but in go it would be much simple coz for 1M goroutines the most OS threads u would have been created could be ur actual CPU core count (will come back later on explaining this), so Go is handling all the context switching, scheduling of running those tasks(cpu_heavy_lifting_work) from the go runtime(userspace) instead of handling those to kernel.

NOTE: Whenever you hear about OS threads, its not like it can spin only as many as your cores, for eg: lets say u have a machine of 16cores, on this machine as well u can have 1M OS threads and the kernel actually do the context switching, scheduling on these real physical cores and determine what needs to run.

How Goroutines work under the hood?#

As said above Goroutine is a feature used by go in order to achieve concurrency or parallelism, on a very high level goroutines works on a M:P:G basis with M as the OS threads and G as the actual Goroutines and P is the important part where it contains actual context + queues etc, and all these are managed by your actual go runtime.

Now consider the below Go code, its a sample psuedo code, it will have syntax issues.

func main(){
    var wg sync.WaitGroup
    cpuWorker := func(wg *sync.WaitGroup) {
        defer wg.Done()
        sleep(60)
    }
    nwWorker := func(wg *sync.WaitGroup) {
        defer wg.Done()
        http.req() 
    }

    wg.Add(5)
    go cpuWorker()
    go cpuWorker()
    go cpuWorker()
    go nwWorker()
    go nwWorker()
    wg.Wait()
}

In the above code we basically spawns 5 goroutines, out of which 3 does some cpu intensive task and 2does network io.

Now lets learn about what are M, G, P mentioned above,

M is basically the Os level thread, its created using syscall by the goruntime and this is the one that is actually responsible for running any goroutine.

G is all our goroutines

P is the context + queues(LRQ) and other stuff, this actually contains the goroutines that needs to be run on M, every M needs a P in order to fetch a G and run it on M, the maximum P is the number of GOMAXPROCS that you configured for your program, by default it is the number of actual cores in your machine.

so in theory go cannot have more Ps then your actual GOMAXPROCS configured, since go is associating every P to an M, so the max M(OS threads) that is doing the actual working is equal to P or sometimes greater, but not huge proportion, so this is reason go doesnt spins up that many number of OS threads(M).

Now lets go through this understanding with the above go code. Now for above code consider GOMAXPROCS is 2, when u compile and run the above code main, first a process will be created and an initial OS thread will be given from kernel, lets say this is M0

From the above screenshot, when u start this code/process main in this case, at the start itself, goruntime creates GOMAXPROCS number of Ps and also the initial thread given by the kernel M0 each thread M contains a code lets say g0, this g0 is responsible for scheduling, context switching and lot of other stuff(this is goruntime userspace work) at the start of the program, we will get only one thread M0(you can also think of as main thread), this will be attached to any of the Ps, in our case lets consider P1.

Now this g0 will first place our main goroutine(yes even main is a goroutine) in its P(which is P1), and starts to run NOTE: still P2 is simply idle without any M(means it just a dumb and cant do any thing)

Now when running the main goroutine G1, the G1 itself from M0 scheduler sees 5 goroutines G2, G3, G4, G5, G6, all these are placed under its own P1 queue(LRQ), and it also checks if atleast one "P" is idle without any M, if this is true, it basically creates a new/allocates an idle M M(M1 in our case) and then attaches that to P2(in our case this is idle) as u can see from the screenshot.

Now g0 from M0 starts poping tasks from its LRQ from P1, now you may have a question, How now M1 attached on P2 can do tasks??

The answer is, g0 scheduler now attached to M1 checks for tasks(routines) in its LRQ(P2) it wont find any, since its not there, it then goes to check the GRQ(will come on how this is used) global queue, and if nothings here also, then it goes to random Ps in our case it goes to P1 and picks the Gs.

Now consider, the M0 that is running a G G2 is calling a network request, now g0 on this M0 this is nonblocking(coz of epoll) so the g0 takes out the G2 and continues its existing scheduling process.

Once the G2 is ready(this is informed by constant polling, u can visit my node working using single thread blog to understand this), any M can pick this(usually Ms constantly checks the network poller as well to pick these kinda Gs).

Now consider, the M0 that is running a G G3 is calling a file io, this is actually a blocking call, now the G3 code(not the g0 scheduler code) will basically detaches this M0 and then attach the P1 in any idle M or a new M(this is one of the situation where your OS threads grow larger than GOMAXPROCS(P)).

Once the G3 file read/write completed, now the G3 code searches for its old P1 is idle or not, to make the M0 attach to its P1 if its not idle, it will go and search for anyother idle P(without an M, like in our very first case) now here comes the GRQ global queue part, if all Ps are busy(basically all have Ms) and none of them are idle, the G3 is now placed into the GRQ queue and this M0 is parked as idle.

This is how Goruntime ensures all our goroutines, runs easily in a decentralised way, the things i have explained are so high level, please refer the official go code to understand more.

TL;DR Go routines work by M:P:G model, where P holds the information about Gs and queues the Gs(goroutines) that are needed to run, the M are the actual OS thread that runs any G, M needs a P inorder to fetch the G from its queue and also to extract some stack information wrt G.