Go Concurrency: Goroutines and Channels in Practice

Go’s concurrency model — goroutines and channels — is deceptively simple. Start a goroutine, send values on channels, receive values. The hard part is using these primitives correctly to build concurrent systems that are fast, correct, and free of deadlocks and race conditions.

This post covers patterns I have learned building production services that process millions of requests concurrently: when to use goroutines, how to structure channels, and the patterns that avoid common pitfalls.

Goroutines: The Lightweight Threads

Goroutines are lightweight — you can run thousands of them on a single machine:

func main() {
    for i := 0; i < 1000; i++ {
        go func(n int) {
            fmt.Printf("Goroutine %d\n", n)
        }(i)
    }

    time.Sleep(time.Second)
}

Key goroutine rules:

1. Always pass loop variables as arguments — otherwise all goroutines share the same variable
2. Know when goroutines exit — they exit when the function returns
3. Handle panics — unhandled panics in goroutines crash the program

// BAD: All goroutines share the same loop variable
for i := 0; i < 10; i++ {
    go func() {
        fmt.Println(i) // All goroutines print the same value!
    }()
}

// GOOD: Pass loop variable as argument
for i := 0; i < 10; i++ {
    go func(n int) {
        fmt.Println(n)
    }(i)
}

// GOOD: Use a local variable
for i := 0; i < 10; i++ {
    i := i // Shadow loop variable
    go func() {
        fmt.Println(i)
    }()
}

Channels: Communicating Sequential Processes

Channels coordinate goroutines by passing values between them:

ch := make(chan int)

// Sender goroutine
go func() {
    ch <- 42 // Send value
    close(ch) // Signal no more values
}()

// Receiver
value := <-ch // Receive value
value, ok := <-ch // Receive with channel-open status

Channel types:

• Unbuffered — make(chan T) — sender blocks until receiver is ready
• Buffered — make(chan T, n) — sender blocks only when buffer is full
• Receive-only — <-chan T — can only receive from
• Send-only — chan<- T — can only send to

// Unbuffered channel — synchronous handoff
unbuffered := make(chan int)
go func() {
    unbuffered <- 1 // Blocks until receiver ready
}()
val := <-unbuffered // Receives immediately

// Buffered channel — async within buffer size
buffered := make(chan int, 10)
buffered <- 1     // Doesn't block (buffer not full)
buffered <- 2     // Doesn't block
val := <-buffered // Receives 1

Pattern: Worker Pool

The worker pool limits concurrency by reusing a fixed number of goroutines:

func workerPool[T any](ctx context.Context, numWorkers int, jobs <-chan T, processor func(context.Context, T) error) error {
    g, ctx := errgroup.WithContext(ctx)

    // Start workers
    for i := 0; i < numWorkers; i++ {
        g.Go(func() error {
            for {
                select {
                case <-ctx.Done():
                    return ctx.Err()
                case job, ok := <-jobs:
                    if !ok {
                        return nil // Jobs channel closed, exit worker
                    }
                    if err := processor(ctx, job); err != nil {
                        return err
                    }
                }
            }
        })
    }

    return g.Wait()
}

// Usage
func ProcessBatch(ctx context.Context, items []Item) error {
    jobs := make(chan Item)

    // Feed jobs
    go func() {
        for _, item := range items {
            jobs <- item
        }
        close(jobs)
    }()

    // Run with 10 workers
    return workerPool(ctx, 10, jobs, func(ctx context.Context, item Item) error {
        return processItem(ctx, item)
    })
}

The worker pool pattern prevents resource exhaustion. Without it, creating a goroutine per item might exhaust memory or database connections.

Pattern: Pipeline

Pipelines transform data through stages — each stage is a goroutine with input and output channels:

// Generator stage — produces data
func generator(nums ...int) <-chan int {
    out := make(chan int)
    go func() {
        for _, n := range nums {
            out <- n
        }
        close(out)
    }()
    return out
}

// Transform stage — squares input
func square(in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        for n := range in {
            out <- n * n
        }
        close(out)
    }()
    return out
}

// Sink stage — consumes output
func print(in <-chan int) {
    for n := range in {
        fmt.Println(n)
    }
}

// Pipeline: generator → square → print
func main() {
    nums := generator(1, 2, 3, 4, 5)
    squared := square(nums)
    print(squared)
}

Real-world pipeline with multiple stages:

func ProcessUsers(ctx context.Context, userIDs []string) <-chan *UserProfile {
    // Stage 1: Fetch users from database
    userCh := fetchUsers(ctx, userIDs)

    // Stage 2: Enrich with profile data
    profileCh := enrichProfiles(ctx, userCh)

    // Stage 3: Filter active users
    activeCh := filterActive(ctx, profileCh)

    return activeCh
}

func fetchUsers(ctx context.Context, ids []string) <-chan *User {
    out := make(chan *User)
    go func() {
        defer close(out)
        for _, id := range ids {
            user, err := db.GetUser(ctx, id)
            if err != nil {
                continue // Skip errors
            }
            select {
            case out <- user:
            case <-ctx.Done():
                return
            }
        }
    }()
    return out
}

Pattern: Fan-In / Fan-Out

Fan-out distributes work across multiple goroutines:

func fanOut(ctx context.Context, in <-chan int, numWorkers int) []<-chan int {
    outs := make([]<-chan int, numWorkers)

    for i := 0; i < numWorkers; i++ {
        outs[i] = worker(ctx, in)
    }

    return outs
}

func worker(ctx context.Context, in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        defer close(out)
        for n := range in {
            result := process(n)
            select {
            case out <- result:
            case <-ctx.Done():
                return
            }
        }
    }()
    return out
}

Fan-in combines results from multiple goroutines:

func fanIn[T any](ctx context.Context, channels ...<-chan T) <-chan T {
    out := make(chan T)

    for _, ch := range channels {
        go func(c <-chan T) {
            for v := range c {
                select {
                case out <- v:
                case <-ctx.Done():
                    return
                }
            }
        }(ch)
    }

    return out
}

Complete fan-out/fan-in example:

func ProcessConcurrently(ctx context.Context, items []Item) ([]Result, error) {
    // Fan-out: distribute work
    jobs := make(chan Item)
    results := make(chan Result)

    // Start workers
    var wg sync.WaitGroup
    for i := 0; i < 10; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for item := range jobs {
                result := processItem(item)
                select {
                case results <- result:
                case <-ctx.Done():
                    return
                }
            }
        }()
    }

    // Feed jobs
    go func() {
        for _, item := range items {
            jobs <- item
        }
        close(jobs)
    }()

    // Wait for workers and close results
    go func() {
        wg.Wait()
        close(results)
    }()

    // Fan-in: collect results
    var allResults []Result
    for result := range results {
        allResults = append(allResults, result)
    }

    return allResults, nil
}

Pattern: Rate Limiting

Rate limiting prevents overwhelming external services:

import "golang.org/x/time/rate"

type RateLimitedClient struct {
    client  *http.Client
    limiter *rate.Limiter
}

func NewRateLimitedClient(rps int) *RateLimitedClient {
    // rps requests per second
    limiter := rate.NewLimiter(rate.Limit(rps), 1) // Burst of 1
    return &RateLimitedClient{
        client:  &http.Client{},
        limiter: limiter,
    }
}

func (c *RateLimitedClient) Do(req *http.Request) (*http.Response, error) {
    // Wait until rate limit allows request
    if err := c.limiter.Wait(req.Context()); err != nil {
        return nil, err
    }

    return c.client.Do(req)
}

For simple rate limiting with a ticker:

func ProcessWithRateLimit(items []Item, requestsPerSecond int) {
    interval := time.Second / time.Duration(requestsPerSecond)
    ticker := time.NewTicker(interval)
    defer ticker.Stop()

    for _, item := range items {
        <-ticker.C // Wait for tick
        go processItem(item)
    }
}

Pattern: Timeout per Operation

When processing concurrently, each operation should have its own timeout:

func processWithTimeout(ctx context.Context, item Item, timeout time.Duration) error {
    ctx, cancel := context.WithTimeout(ctx, timeout)
    defer cancel()

    resultCh := make(chan error, 1)

    go func() {
        resultCh <- processItem(ctx, item)
    }()

    select {
    case err := <-resultCh:
        return err
    case <-ctx.Done():
        return fmt.Errorf("operation timed out: %w", ctx.Err())
    }
}

func ProcessBatch(ctx context.Context, items []Item) error {
    g, ctx := errgroup.WithContext(ctx)

    for _, item := range items {
        item := item
        g.Go(func() error {
            return processWithTimeout(ctx, item, 5*time.Second)
        })
    }

    return g.Wait()
}

Common Pitfalls

Goroutine Leaks

Forgetting to close channels or handle cancellation causes leaks:

// BAD: Goroutine leak — channel never closed
func leaky() {
    ch := make(chan int)
    go func() {
        val := <-ch // Goroutine blocks forever
        fmt.Println(val)
    }()
    // Function returns, goroutine waits forever
}

// GOOD: Use context or close channel
func nonLeaky(ctx context.Context) {
    ch := make(chan int)
    go func() {
        select {
        case val := <-ch:
            fmt.Println(val)
        case <-ctx.Done():
            return // Clean exit
        }
    }()
}

Deadlocks

Deadlocks occur when goroutines wait on each other:

// DEADLOCK: Sending on unbuffered channel with no receiver
func deadlock() {
    ch := make(chan int)
    ch <- 42 // Blocks forever — no receiver
}

// DEADLOCK: Waiting for own send
func deadlock2() {
    ch := make(chan int)
    go func() {
        ch <- <-ch // Waits for receive, but needs send first
    }()
}

// GOOD: Use goroutine or buffered channel
func noDeadlock() {
    ch := make(chan int, 1) // Buffered
    ch <- 42
    val := <-ch
    fmt.Println(val)
}

Race Conditions

Use go run -race to detect data races:

// RACE CONDITION: Multiple goroutines access shared variable
func race() {
    var counter int
    for i := 0; i < 1000; i++ {
        go func() {
            counter++ // Data race!
        }()
    }
}

// GOOD: Use mutex or atomic
func noRace() {
    var counter int64
    for i := 0; i < 1000; i++ {
        go func() {
            atomic.AddInt64(&counter, 1)
        }()
    }
}

Closing Closed Channels

Closing an already-closed channel panics:

// BAD: May panic if called multiple times
func unsafeClose(ch chan int) {
    close(ch)
}

// GOOD: Use sync.Once
type SafeChannel struct {
    ch chan int
    once sync.Once
}

func (sc *SafeChannel) Close() {
    sc.once.Do(func() {
        close(sc.ch)
    })
}

Select for Timeout and Cancellation

The select statement handles multiple channel operations:

select {
case data := <-dataCh:
    // Process data
case err := <-errCh:
    // Handle error
case <-time.After(5 * time.Second):
    // Timeout
case <-ctx.Done():
    // Context cancelled
}

Non-blocking select:

select {
case val := <-ch:
    fmt.Println("received:", val)
default:
    fmt.Println("no value available")
}

Key Takeaways

1. Use worker pools to limit concurrency — don’t spawn unlimited goroutines
2. Pass loop variables as arguments — avoid closures capturing shared variables
3. Buffer channels for async, unbuffered for sync — choose based on coordination needs
4. Always check for context cancellation — prevent goroutine leaks
5. Use select for timeout and cancellation — handle multiple channels
6. Detect races with -race flag — run tests with race detection
7. Close channels once — use sync.Once if needed
8. Prefer errgroup for concurrent operations — handles cancellation and error aggregation

Go concurrency primitives are simple, but correct usage requires discipline. The patterns in this post — worker pools, pipelines, fan-in/fan-out, and proper error handling — form the foundation of production-grade concurrent systems.