Suddenly got below errors which killed my daemon:
fatal error: concurrent map writes goroutine 646 [running]: runtime.throw(0x75fd38, 0x15) /usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc000315e60 sp=0xc000315e30 pc=0x42ecf2 runtime.mapdelete_fast64(0x6f0800, 0xc00008ad50, 0x2b3e) goroutine 1 [sleep]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 time.Sleep(0x12a05f200) /usr/local/go/src/runtime/time.go:105 +0x157 webhook/worker.Manager() goroutine 6 [IO wait]: internal/poll.runtime_pollWait(0x7fc308de6f08, 0x72, 0x0) /usr/local/go/src/runtime/netpoll.go:184 +0x55 internal/poll.(*pollDesc).wait(0xc000110018, 0x72, 0x0, 0x0, 0x75b00b) /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Accept(0xc000110000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8 net.(*netFD).accept(0xc000110000, 0xc000050d50, 0xc000046700, 0x7fc308e426d0) /usr/local/go/src/net/fd_unix.go:238 +0x42 net.(*TCPListener).accept(0xc000126000, 0xc000050d80, 0x40dd08, 0x30) /usr/local/go/src/net/tcpsock_posix.go:139 +0x32 net.(*TCPListener).Accept(0xc000126000, 0x72f560, 0xc0000f0180, 0x6f4f20, 0x9c00c0) /usr/local/go/src/net/tcpsock.go:261 +0x47 net/http.(*Server).Serve(0xc0000f4000, 0x7ccbe0, 0xc000126000, 0x0, 0x0) /usr/local/go/src/net/http/server.go:2896 +0x286 net/http.(*Server).ListenAndServe(0xc0000f4000, 0xc0000f4000, 0x8) /usr/local/go/src/net/http/server.go:2825 +0xb7 net/http.ListenAndServe(...) /usr/local/go/src/net/http/server.go:3080 webhook/handler.HandleRequest()
In starting for a few seconds it was working smoothly.
After few seconds my service got kill with above mentioned error.
Initialized one global variable with the type ‘map’. Where the key is
int and value is
var ActiveInstances = make(map[int](chan string))
Having two functions:
go SetValue() go DeleteValue()
ActiveInstances[id] = make(chan string, 5)
Both functions running in multiple goroutines.
The error itself says
concurrent map writes, By which I got the idea that something is wrong with my map variable
ActiveInstances. Both functions in multiple goroutines are trying to access the same variable(
ActiveInstances) at the same time. Which created race condition.
After exploring a few blogs & documentation, I become to know that “Maps are not safe for concurrent use”.
As per golang doc
Map access is unsafe only when updates are occurring. As long as all goroutines are only reading—looking up elements in the map, including iterating through it using a for range loop—and not changing the map by assigning to elements or doing deletions, it is safe for them to access the map concurrently without synchronization.golang
Here we need to access
ActiveInstances synchronously. We want to make sure only one goroutine can access a variable at a time to avoid conflicts, This can be easily achieved by
sync.Mutex. This concept is called mutual exclusion which provides methods
We can define a block of code to be executed in mutual exclusion by surrounding it with a call to
Unlock It is as simple as below:
mutex.Lock() //my block of code mutex.Unlock()
mutex.Lock() ActiveInstances[i_id] = make(chan string, 5) mutex.Unlock() mutex.Lock() delete(ActiveInstances, id) mutex.Unlock()
This is how we successfully fix this problem.