Published
- 3 min read
sync/singleflight
Background
Duplicated workloads can be redundant and could even be fatal in some cases.
sync/singleflight provides the mechanism to deduplicate workloads.
ELI5 Why we need it
Imagine a family of 4 that loves to watch Netflix.
One day the television broke down and everyone hurried to replace it.
In the end, the family ended up with 4 televisions, not exactly ideal.
This is where sync/singleflight comes into the spotlight.
Having sync/singleflight is like having a person of authority where every request has to pass through him/her and the person of authority decides who gets to execute it.
In a family, the person of authority could be the father/mother.
When the television breaks down, everybody checks with the person of authority if he/she can replace the television, and only one person is allowed to purchase the television and share the television with the entire family.
How to use it
First, let us define a function to mimic a family member buying a new television(TV):
// buyNewTV buys a new TV and returns its ID
func buyNewTV() (int32, error) {
time.Sleep(1*time.Second)
fmt.Println("Bought a new TV!")
return rand.Int31n(1000), nil
}Now let’s see what happens if the family tries to replace the TV without sync/singleflight:
func replaceTVWithoutSingleFlight() {
wg := errgroup.Group{} // spot something new? check out the post on sync/errgroup.
for i := 0; i < 4; i++ {
wg.Go(func() error {
tvID, err := buyNewTV()
if err != nil {
return err
}
fmt.Println(tvID)
return nil
})
}
wg.Wait()
}
// Output
// Bought a new TV!
// 57
// Bought a new TV!
// 111
// Bought a new TV!
// 998
// Bought a new TV!
// 713VS what happens with sync/singleflight:
func replaceTVWithSingleFlight() {
sfg := singleflight.Group{} // init singleflight group
wg := errgroup.Group{}
for i := 0; i < 4; i++ {
wg.Go(func() error {
// execute functions via sfg.Do
// functions are deduplicated via "buyNewTV" key
tvID, err, _ := sfg.Do("buyNewTV", func() (interface{}, error) {
return buyNewTV()
})
if err != nil {
return err
}
fmt.Println(tvID)
return nil
})
}
wg.Wait()
}
// Output
// Bought a new TV!
// 942
// 942
// 942
// 942Awesome, the family only bought 1 TV and everyone gets to enjoy it.
Over here 4 goroutines attempted to buy a new TV, however, they all had the same purpose/key buyNewTV hence singleflight group only allowed 1 goroutine to proceed and the rest waited until the result was ready.
Rabbit Hole
sync/singleflight provides the mechanism to deduplicate workloads.
This could be great when:
- An authentication token expires and multiple goroutines rush to obtain new a token.
- A multiplexed connection disconnects and multiple goroutines rush to reconnect.
- Multiple routines trying to call a costly external API that could be deduplicated.
*There could be other well-suited solutions for the above scenarios as well, so just treat this as a tool/option.
However, do be careful if the function returns a pointer. The results have become a shared pointer but everyone thinks it belongs to them.
Just like how everyone believes the TV belongs to them and fights to watch their favorite channels/shows.
Alright, that is all for today. Let’s get one post smarter every day, and see you tomorrow!