Concurrency Patterns: Fan-in and Fan-out in Go

Introduction

Concurrency patterns are essential in modern programming to efficiently manage parallel tasks. In Go, two important concurrency patterns are Fan-in and Fan-out. These patterns help manage and coordinate multiple goroutines effectively.

Fan-out

Fan-out refers to the process of starting multiple concurrent tasks from a single point. In Go, this often means spawning multiple goroutines to perform tasks concurrently.

func main() {
    jobs := make(chan int, 10)
    results := make(chan int, 10)

    for w := 1; w <= 3; w++ {
        go worker(w, jobs, results)
    }

    for j := 1; j <= 9; j++ {
        jobs <- j
    }
    close(jobs)

    for a := 1; a <= 9; a++ {
        <-results
    }
}

func worker(id int, jobs <-chan int, results chan<- int) {
    for j := range jobs {
        fmt.Println("worker", id, "started  job", j)
        time.Sleep(time.Second)
        fmt.Println("worker", id, "finished job", j)
        results <- j * 2
    }
}
                    

In the example above, we have a function worker that takes jobs from the jobs channel. We spawn three worker goroutines to process jobs concurrently, demonstrating the Fan-out pattern.

Fan-in

Fan-in is the opposite of Fan-out. It involves collecting results from multiple concurrent tasks into a single point. In Go, this typically means gathering output from multiple goroutines into a single channel.

func main() {
    c1 := make(chan string)
    c2 := make(chan string)

    go func() {
        time.Sleep(time.Second * 1)
        c1 <- "one"
    }()
    go func() {
        time.Sleep(time.Second * 2)
        c2 <- "two"
    }()

    for i := 0; i < 2; i++ {
        select {
        case msg1 := <-c1:
            fmt.Println("Received", msg1)
        case msg2 := <-c2:
            fmt.Println("Received", msg2)
        }
    }
}
                    

In this example, we have two goroutines that send messages to channels c1 and c2. The select statement is used to receive messages from both channels, demonstrating the Fan-in pattern.

Combining Fan-in and Fan-out

Often, you will need to combine both Fan-in and Fan-out patterns to efficiently manage concurrent tasks. This involves spawning multiple goroutines (Fan-out) and then collecting their results (Fan-in).

func main() {
    jobs := make(chan int, 100)
    results := make(chan int, 100)

    for w := 1; w <= 3; w++ {
        go worker(w, jobs, results)
    }

    for j := 1; j <= 5; j++ {
        jobs <- j
    }
    close(jobs)

    for a := 1; a <= 5; a++ {
        fmt.Println(<-results)
    }
}

func worker(id int, jobs <-chan int, results chan<- int) {
    for j := range jobs {
        fmt.Println("worker", id, "started  job", j)
        time.Sleep(time.Second)
        fmt.Println("worker", id, "finished job", j)
        results <- j * 2
    }
}
                    

Here, jobs are distributed among three workers (Fan-out), and their results are collected in the results channel (Fan-in). This pattern is useful for parallel processing tasks and aggregating their results efficiently.

Conclusion

Understanding and implementing Fan-in and Fan-out patterns in Go can significantly enhance the efficiency and scalability of your programs. These patterns allow you to manage multiple concurrent tasks and aggregate their results seamlessly.