Overview

Yes, I do mean race, not static, and Go is popular for its simplicity and great concurrent development experience, but we often write code with Data Race, but Go can often help us check it out, and this article will try to explain how Go does it.

What is Data Race

In concurrent programming, data race may occur when multiple threads, threads or processes read or write to a memory area at the same time. Data race is when two or more threads access the same shared variable at the same time without reasonable synchronization, and at least one thread is writing to the value of the variable. If these accesses are by different threads and at least one of the accesses is a write access, then the situation is called data race.

Data contention can lead to unpredictable program behavior, including crashes, deadlocks, infinite loops, etc. Therefore, it is important to avoid data contention as a problem in concurrent programming. A common solution is to use synchronization mechanisms, such as locks, semaphores, etc., to ensure that access to shared variables by different threads is ordered.

Go detects data contention

Go’s own tool chain provides several tools to detect data contention, namely

• Detecting Data Race at test time: go test -race mypkg
• Detecting Data Race at build time: go build -race mycmd
• Detecting Data Race at install time: go install -race mypkg
• Detects Data Race at runtime: go run -race mysrc.go

When he detects a Data Race, he reports an error in this format:

[root@liqiang.io]# go test -race liqiang.io/liuliqiang/testpackage
WARNING: DATA RACE
Read by goroutine 185:
  net.(*pollServer).AddFD()
      src/net/fd_unix.go:89 +0x398
  net.(*pollServer).WaitWrite()
      src/net/fd_unix.go:247 +0x45
  net.(*netFD).Write()
      src/net/fd_unix.go:540 +0x4d4
  net.(*conn).Write()
      src/net/net.go:129 +0x101
  net.func·060()
      src/net/timeout_test.go:603 +0xaf
Previous write by goroutine 184:
  net.setWriteDeadline()
      src/net/sockopt_posix.go:135 +0xdf
  net.setDeadline()
      src/net/sockopt_posix.go:144 +0x9c
  net.(*conn).SetDeadline()
      src/net/net.go:161 +0xe3
  net.func·061()
      src/net/timeout_test.go:616 +0x3ed
Goroutine 185 (running) created at:
  net.func·061()
      src/net/timeout_test.go:609 +0x288
Goroutine 184 (running) created at:
  net.TestProlongTimeout()
      src/net/timeout_test.go:618 +0x298
  testing.tRunner()
      src/testing/testing.go:301 +0xe8

Although there is a lot of content, there are usually only two keywords you need to look at:

• Read by goroutine: this indicates who the code is that reads the same block of memory when a Data Race occurs;
• Previous write by goroutine: this indicates who is the code that writes the same block of memory when a Data Race occurs;

Common solutions

This allows you to analyze why Data Race occurs. Once you have analyzed the cause, the solutions I have used are

• Use synchronization mechanisms: This is the simplest solution, you can directly use synchronization mechanisms, such as locks, semaphores, etc.; synchronization mechanisms can ensure that access to shared variables by different threads is in order, thus avoiding Data Race problems.
• Use atomic operations: Atomic operations are special operations that ensure that access to shared variables is atomic, thus avoiding the Data Race problem. For example, Go comes with the Atomic data type: eg: atomic.Uint64 and my usual extension to go.uber.org/atomic: atomic.String, etc.
• Use concurrency-safe data structures: Some data structures, such as thread-safe queues, hash tables, etc., can avoid Data Race problems. When using these data structures, care should be taken to avoid multiple threads modifying the same node or element at the same time, which is actually a synchronization mechanism, but abstracted by the data type.

How Go is implemented

Once we know what a Data Race is and how it is commonly solved, let’s take a look at how Go detects Data Races. According to the definition of Data Race, we know that in order for a Data Race to occur, there must be two people reading and writing to the same memory at the same time, during which there is a crossover process, so Go actually implements it by this principle.

Go detects Data Race in a way similar to locking, that is, it punches before and after the memory access, for example, in this original code:

[root@liqiang.io]# cat raw.go
func main() {
    go func() {
        x = 1
    }()
    fmt.Println(x)
}

When Data Race (-race) is turned on, the Go-generated code may look like:

[root@liqiang.io]# cat compile.go
func main() {
    go func() {
        // Notice Race Detector there will be a write operation
        race.WriteAcquire(&x)
        x = 1
        // Notice Race Detector the write operation will be finish
        race.WriteRelease(&x)
    }()
    // Notice Race Detector there will be a read operation
    race.ReadAcquire(&x)
    value := x
    // Notice Race Detector the read operation will be finish
    race.ReadRelease(&x)
    fmt.Println(value)
}

It is equivalent to punting before and after memory operations, and then there is a component dedicated to detecting Data Race: Data Race Detector, which can be used to detect if there are conflicting accesses to the same block of memory, and the overall process is

1. Race Detector uses a data structure called Shadow Memory to store the metadata of memory accesses. For each memory address, Shadow Memory records the two most recent access operations, including the type of operation (read or write), the goroutine of the operation, and the moment when the operation occurred.
2. Check for concurrent accesses: When Race Detector detects a memory access operation, it checks the Shadow Memory records associated with that operation. If one of the following conditions is found, then data contention is considered to exist:
   • The current operation is a write operation and occurs concurrently (i.e., there is no happens-before relationship) with one of the two most recent access operations (either read or write), and the two access operations are from different Goroutines.
   • The current operation is a read operation and occurs concurrently with one of the most recent write operations (i.e., there is no happens-before relationship), and the two operations are from different Goroutines. 3.
3. Report data contention: When data contention is detected, Race Detector generates a detailed report including information about where the data contention occurred, the Goroutine involved, and the stack trace.

Different memory handling

The above mentioned is a general idea, but we know that Go has different types of memory, such as the simplest heap memory and stack memory, which have different access domains and different triggering conditions for Data Race (for example, heap memory will have a higher probability of triggering, while stack memory will have a low probability). The following summarizes the different ways of handling different kinds of memory:

• Global variables and heap memory: for global variables and memory allocated on the heap, the compiler usually inserts data race detection code for all read and write operations, this is because global variables and heaps are shared between multiple Goroutines within them, so they are more prone to data races;
• Stack memory: For stack memory (e.g., local variables), the compiler may adopt a different treatment strategy. Since stack memory is usually local to the Goroutine and has an independent lifecycle between function calls, in many cases stack memory accesses are not prone to data contention. However, when the address of stack memory is shared with other Goroutines (e.g., through pointer passing or closure capture), the compiler needs to insert data contention detection code for these memory accesses.
• Optimization and Elimination: The compiler may optimize or eliminate data contention detection for certain memory accesses. For example, the compiler may analyze the static information and runtime behavior of the code to identify certain memory accesses where data contention is not likely to occur, and thus avoid inserting detection code for those accesses. This optimization can reduce performance loss, but may also lead to incomplete data contention detection in some extreme cases.
• Atomic operations and synchronous primitives: The compiler inserts special detection code for atomic operations (e.g. atomic.AddInt32, atomic.LoadUint64, etc.) and synchronous primitives (e.g. Mutex, RWMutex, Channel, etc.). These codes help Race Detector to analyze the synchronization behavior of the program more accurately and thus detect data contention more reliably.

Data Race’s goroutine model

Race Detector is tightly integrated with the Go runtime in order to detect data contention in real time during program execution. At runtime, Race Detector’s various functions are distributed across multiple Goroutines and threads. I have summarized some of the more commonly used components below:

• Stub Code: As mentioned earlier, the Go compiler inserts additional code at memory access and synchronization operations to communicate with the Race Detector. This inserted code is executed during program runtime, directly in the Goroutine where the memory accesses and synchronization operations occur.
• Shadow Memory: Race Detector uses Shadow Memory to store metadata for memory accesses. shadow Memory is managed and updated in the Goroutine where the memory access operation occurs, to ensure real-time metadata.
• Data contention detection: Race Detector’s data contention detection logic is usually performed in the Goroutine where the memory access operation occurs. This means that when a Goroutine performs a memory access operation, Race Detector checks for data contention in the same Goroutine.
• Reporting and Diagnostics: When Race Detector detects data contention, it generates a detailed report including information about where the data contention occurred, the Goroutines involved, and the stack trace. Report generation may involve multiple Goroutines and threads, as it needs to collect and organize various contextual information.

We can expand on point 4 of this, as it also involves the coordination of multiple goroutines:

1. When a memory access operation occurs for a Goroutine, Race Detector checks if other Goroutines competing with it exist. This is done by analyzing the metadata in Shadow Memory, which contains the access history and synchronization relationships for each memory address. 2.
2. If data contention is detected, Race Detector collects information about the competing Goroutine, including its ID, stack trace, the memory address where the contention occurred, and the associated source code location. 3.
3. Race Detector can detect multiple data contention events at the same time. For each event, it generates a separate report. The report will contain detailed information about the competing Goroutine to help developers understand and resolve the problem. 4.
4. Finally, when program execution ends or when a data contention is detected, Race Detector prints all the reports collected to the standard error output (stderr). Each report is displayed individually and in the order in which it occurred. This allows developers to view and analyze data contention events one by one.

Summary

In this article, I have described the use and rationale for Go’s Data Race detection mechanism. It is important to note that while Go’s Data Race is easy to enable and the Go compiler and Race Detector will detect all data contentions for memory accesses whenever possible, they cannot guarantee 100% accuracy and completeness. Therefore, developers still need to follow good programming practices to ensure that synchronization and merging of programs is done correctly and safely.

And as you can see from the previous introduction, adding Data Race detection will definitely increase memory usage and decrease execution speed, according to the official documentation: memory usage will increase by 5-10 times and execution time will decrease by 2-20 times.

Ref

• Data Race Detector
• Race Detector Code