Matt Pearring and Alex Rakoczy 16 August 2021
Today the Go team is thrilled to release Go 1.17, which you can get by visiting the download page.
This release brings additional improvements to the compiler, namely a new way of passing function arguments and results. This change has shown about a 5% performance improvement in Go programs and reduction in binary sizes of around 2% for amd64 platforms. Support for more platforms will come in future releases.
Go 1.17 also adds support for the 64-bit ARM architecture on Windows, letting gophers run Go natively on more devices.
We’ve also introduced pruned module graphs in this release. Modules that specify go 1.17 or higher in their go.mod file will have their module graphs include only the immediate dependencies of other Go 1.17 modules, not their full transitive dependencies. This should help avoid the need to download or read go.mod files for otherwise irrelevant dependencies—saving time in everyday development.
go 1.17
go.mod
Go 1.17 comes with three small changes to the language. The first two are new functions in the unsafe package to make it simpler for programs to conform to the unsafe.Pointer rules: unsafe.Add allows for safer pointer arithmetic, while unsafe.Slice allows for safer conversions of pointers to slices. The third change is an extension to the language type conversion rules to allow conversions from slices to array pointers, provided the slice is at least as large as the array at runtime.
unsafe
unsafe.Pointer
unsafe.Add
unsafe.Slice
Finally there are quite a few other improvements and bug fixes, including verification improvements to crypto/x509, and alterations to URL query parsing. For a complete list of changes and more information about the improvements above, see the full release notes.
Thanks to everyone who contributed to this release by writing code, filing bugs, sharing feedback, and testing the beta and release candidates. Your efforts helped to ensure that Go 1.17 is as stable as possible. As always, if you notice any problems, please file an issue.
We hope you enjoy the new release!
Steve Francia 23 June 2021
Since the earliest days of Go, Stack Overflow has been a significant part of the Go user experience. For the past five years, the Go user survey has consistently identified Stack Overflow as the #1 place users go to find answers to their Go questions.
We are excited to share the Go Collective, the very first open source project in Collectives™ on Stack Overflow, which came as a result of a partnership between Go & Stack Overflow.
Stack Overflow’s mission for the community is to help make the most out of Stack Overflow as a tool for knowledge reusability. For the millions of users in the Go community, the Go Collective will be an improved Stack Overflow experience. It will retain the great question and answer experience we’ve all come to appreciate from Stack Overflow. But it will also provide a home for gophers and an opportunity for Go community leaders to have a voice and help establish best practices.
The benefits of the Go Collective will be:
Find trusted answers faster: we understand developers often have to test multiple solutions to a problem before finding the right one. With the Go Collective, you’ll now see answers that are recommended by the Go team or provided by Recognized Members, meaning Go team members, Go Google Developer Experts, and Stack Overflow users who have been recognized by Stack Overflow as subject matter experts on Go ecosystem technologies. These users will have a badge identifying them as Recognized Members when they post or edit a question, provide an answer, or write an article.
Get product information all in one place: the Go Collectives page on Stack Overflow centralizes all of the questions and answers and content associated with Go and related tags. There will be links from the collective to critical Go websites, and Go websites will direct viewers to the Go Collective on Stack Overflow.
Explore new content formats: stay up-to-date on our technologies and find more in-depth product information with articles, long-form content like how-to guides and announcements that can be found on the Go Collective page.
Earn recognition: Collectives on Stack Overflow also gives us a way to recognize you for your contributions to the community. There’s a leaderboard you can access via the Go Collective (see the “Members” tab) that will help identify trusted contributors designated as Recognized Members. If you are interested in becoming a Recognized Member, please email stackoverflow@golang.org.
Today Google Cloud is launching their own collective alongside the Go Collective. We are excited to take this step together: Go is the best language for building cloud infrastructure and applications, and Google Cloud is the best place to run Go applications. And now Stack Overflow is the best place to find answers to your Go and Cloud questions.
To read more about this see Stack Overflow's blog post and Google Cloud's blog post.
Katie Hockman and Jay Conrod 3 June 2021
We are excited to announce that native fuzzing is ready for beta testing in its development branch, dev.fuzz!
Fuzzing is a type of automated testing which continuously manipulates inputs to a program to find issues such as panics or bugs. These semi-random data mutations can discover new code coverage that existing unit tests may miss, and uncover edge case bugs which would otherwise go unnoticed. Since fuzzing can reach these edge cases, fuzz testing is particularly valuable for finding security exploits and vulnerabilities.
See golang.org/s/draft-fuzzing-design for more details about this feature.
To get started, you may run the following
$ go get golang.org/dl/gotip $ gotip download dev.fuzz
This builds the Go toolchain from the dev.fuzz development branch, and won’t be needed once the code is merged to the master branch in the future. After running this, gotip can act as a drop-in replacement for the go command. You can now run commands like
gotip
go
$ gotip test -fuzz=FuzzFoo
There will be ongoing development and bug fixes in the dev.fuzz branch, so you should regularly run gotip download dev.fuzz to use the latest code.
gotip download dev.fuzz
For compatibility with released versions of Go, use the gofuzzbeta build tag when committing source files containing fuzz targets to your repository. This tag is enabled by default at build-time in the dev.fuzz branch. See the go command documentation about build tags if you have questions about how to use them.
// +build gofuzzbeta
A fuzz target must be in a *_test.go file as a function in the form FuzzXxx. This function must be passed a *testing.F argument, much like a *testing.T argument is passed to a TestXxx function.
FuzzXxx
*testing.F
*testing.T
TestXxx
Below is an example of a fuzz target that’s testing the behavior of the net/url package.
// +build gofuzzbeta package fuzz import ( "net/url" "reflect" "testing" ) func FuzzParseQuery(f *testing.F) { f.Add("x=1&y=2") f.Fuzz(func(t *testing.T, queryStr string) { query, err := url.ParseQuery(queryStr) if err != nil { t.Skip() } queryStr2 := query.Encode() query2, err := url.ParseQuery(queryStr2) if err != nil { t.Fatalf("ParseQuery failed to decode a valid encoded query %s: %v", queryStr2, err) } if !reflect.DeepEqual(query, query2) { t.Errorf("ParseQuery gave different query after being encoded\nbefore: %v\nafter: %v", query, query2) } }) }
You can read more about the fuzzing APIs with go doc
gotip doc testing gotip doc testing.F gotip doc testing.F.Add gotip doc testing.F.Fuzz
This is a beta release in a development branch, so you should expect some bugs and an incomplete feature set. Check the issue tracker for issues labelled “fuzz” to stay up-to-date on existing bugs and missing features.
Please be aware that fuzzing can consume a lot of memory and may impact your machine’s performance while it runs. go test -fuzz defaults to running fuzzing in $GOMAXPROCS processes in parallel. You may lower the number of processes used while fuzzing by explicitly setting the -parallel flag with go test. Read the documentation for the go test command by running gotip help testflag if you want more information.
go test -fuzz
$GOMAXPROCS
-parallel
go test
gotip help testflag
Also be aware that the fuzzing engine writes values that expand test coverage to a fuzz cache directory within $GOCACHE/fuzz while it runs. There is currently no limit to the number of files or total bytes that may be written to the fuzz cache, so it may occupy a large amount of storage (ie. several GBs). You can clear the fuzz cache by running gotip clean -fuzzcache.
$GOCACHE/fuzz
gotip clean -fuzzcache
This feature will not be available in the upcoming Go release (1.17), but there are plans to land this in a future Go release. We hope that this working prototype will allow Go developers to start writing fuzz targets and provide helpful feedback about the design in preparation for a merge to master.
If you experience any problems or have an idea for a feature request, please file an issue.
For discussion and general feedback about the feature, you can also participate in the #fuzzing channel in Gophers Slack.
Happy fuzzing!
Alice Merrick 9 March 2021
In 2020, we had another great turnout with 9,648 responses, about as many as 2019. Thank you for putting in the time to provide the community with these insights on your experiences using Go!
You may notice some questions have smaller sample sizes ("n=") than others. That's because some questions were shown to everyone while others were only shown to a random subset of respondents.
Demographic questions help us distinguish which year-over-year differences may result from changes in who responded to the survey versus changes in sentiment or behavior. Because our demographics are similar to last year, we can be reasonably confident that other year-over-year changes aren't primarily due to demographic shifts.
For example, the distribution of organization sizes, developer experience, and industries remained about the same from 2019 to 2020.
Almost half (48%) of respondents have been using Go for less than two years. In 2020, we had fewer responses from those using Go for less than a year.
Majorities said they use Go at work (76%) and outside of work (62%). The percentage of respondents using Go at work has been trending up each year.
This year we introduced a new question on primary job responsibilities. We found that 70% of respondents’ primary responsibility is developing software and applications, but a significant minority (10%) are designing IT systems and architectures.
As in prior years, we found that most respondents are not frequent contributors to Go open-source projects, with 75% saying they do so "infrequently" or "never".
As in prior years, the vast majority of survey respondents reported working with Go on Linux (63%) and macOS (55%) systems. The proportion of respondents who primarily develop on Linux appears to be slightly trending down over time.
For the first time, editor preferences appear to have stabilized: VS Code remains the most preferred editor (41%), with GoLand a strong second (35%). Together these editors made up 76% of responses, and other preferences did not continue to decrease as they had in previous years.
This year we asked respondents to prioritize improvements to their editor by how much they would hypothetically spend if they had 100 “GopherCoins” (a fictional currency). Code completion received the highest average number of GopherCoins per respondent. Half of respondents gave the top 4 features (code completion, navigating code, editor performance and refactoring) 10 or more coins.
A majority of respondents (63%) spend 10–30% of their time refactoring, suggesting that this is a common task and we want to investigate ways to improve it. It also explains why refactoring support was one of the most-funded editor improvements.
Last year we asked about specific developer techniques and found that almost 90% of respondents were using text logging for debugging, so this year we added a follow-up question to find out why. Results show that 43% use it because it allows them to use the same debugging strategy across different languages, and 42% prefer to use text logging over other debugging techniques. However, 27% don't know how to get started with Go's debugging tools and 24% have never tried using Go's debugging tools, so there's an opportunity to improve the debugger tooling in terms of discoverability, usability and documentation. Additionally, because a quarter of respondents have never tried using debugging tools, pain points may be underreported.
For the first time, this year we asked about overall satisfaction. 92% of respondents said they were very or somewhat satisfied using Go during the past year.
This is the 3rd year we've asked the "Would you recommend…" Net Promoter Score (NPS) question. This year our NPS result is a 61 (68% "promoters" minus 6% "detractors"), statistically unchanged from 2019 and 2018.
Similar to previous years, 91% of respondents said they would prefer to use Go for their next new project. 89% said Go is working well for their team. This year we saw an increase in respondents who agreed that Go is critical to their company’s success from 59% in 2019 to 66% in 2020. Respondents working at organizations of 5,000 or more employees were less likely to agree (63%), while those at smaller organizations were more likely to agree (73%).
Like last year, we asked respondents to rate specific areas of Go development according to satisfaction and importance. Satisfaction with using cloud services, debugging, and using modules (areas that last year were highlighted as opportunities for improvement) increased while most importance scores remained about the same. We also introduced a couple new topics: API and Web frameworks. We see that web frameworks satisfaction is lower than other areas (64%). It wasn't as critically important to most current users (only 28% of respondents said it was very or critically important), but it could be a missing critical feature for would-be Go developers.
81% of respondents said they felt very or extremely productive using Go. Respondents at larger organizations were more likely to feel extremely productive than those at smaller organizations.
We’ve heard anecdotally that it’s easy to become productive quickly with Go. We asked respondents who felt at least slightly productive how long it took them to become productive. 93% said it took less than one year, with the majority feeling productive within 3 months.
Although about the same as last year, the percentage of respondents who agreed with the statement "I feel welcome in the Go community" appears to be trending down over time, or at least not holding to the same upward trends as other areas.
We've also seen a significant year-over-year increase in the proportion of respondents who feel Go’s project leadership understands their needs (63%).
All of these results show a pattern of higher agreement correlated with increased Go experience, beginning at about two years. In other words, the longer a respondent has been using Go, the more likely they were to agree with each of these statements.
We asked an open text question on what we could do to make the Go community more welcoming and the most common recommendations (21%) were related to different forms of or improvements/additions to learning resources and documentation.
Building API/RPC services (74%) and CLIs (65%) remain the most common uses of Go. We don't see any significant changes from last year, when we introduced randomization into the ordering of options. (Prior to 2019, options towards the beginning of the list were disproportionately selected.) We also broke this out by organization size and found that respondents use Go similarly at large enterprises or smaller organizations, although large orgs are a little less likely to use Go for web services returning HTML.
This year we now have a better understanding of which kinds of software respondents write in Go at home versus at work. Although web services returning HTML is the 4th most common use case, this is due to non-work related use. More respondents use Go for automation/scripts, agents and daemons, and data processing for work than web services returning HTML. A greater proportion of the least common uses (desktop/GUI apps, games, and mobile apps) are being written outside of work.
Another new question asked how satisfied respondents were for each use case. CLIs had the highest satisfaction, with 85% of respondents saying they were very, moderately or slightly satisfied using Go for CLIs. Common uses for Go tended to have higher satisfaction scores, but satisfaction and popularity don’t perfectly correspond. For example, agents and daemons has 2nd highest proportion of satisfaction but it’s 6th in usage.
Additional follow-up questions explored different use cases, for example, which platforms respondents target with their CLIs. It's not surprising to see Linux (93%) and macOS (59%) highly represented, given the high developer use of Linux and macOS and high Linux cloud usage), but even Windows is targeted by almost a third of CLI developers.
A closer look at Go for data processing showed that Kafka is the only broadly adopted engine, but a majority of respondents said they use Go with a custom data processing engine.
We also asked about larger areas in which respondents work with Go. The most common area by far was web development (68%), but other common areas included databases (46%), DevOps (42%) network programming (41%) and systems programming (40%).
Similar to last year, we found that 76% of respondents evaluate the current Go release for production use, but this year we refined our time scale and found that 60% begin evaluating a new version before or within 2 months of release. This highlights the importance for platform-as-a-service providers to quickly support new stable releases of Go.
This year we found near-universal adoption for Go modules, and a significant increase in the proportion of respondents who only use modules for package management. 96% of respondents said they were using modules for package management, up from 89% last year. 87% of respondents said they were using only modules for package management, up from 71% last year. Meanwhile, the use of other package management tools has decreased.
Satisfaction with modules also increased from last year. 77% of respondents said they were very, moderately or slightly satisfied with modules, compared to 68% in 2019.
Most respondents said they struggle with official documentation. 62% of respondents struggle to find enough information to fully implement a feature of their application and over a third have struggled to get started with something they haven’t done before.
The most problematic areas of official documentation were on using modules and CLI development, with 20% of respondents finding modules documentation slightly or not at all helpful, and 16% for documentation around CLI development.
Go was designed with modern distributed computing in mind, and we want to continue to improve the developer experience of building cloud services with Go.
Respondents deploying to AWS and Azure saw increases in deploying to a managed Kubernetes platform, now at 40% and 54%, respectively. Azure saw a significant drop in the proportion of users deploying Go programs to VMs and some growth in container usage from 18% to 25%. Meanwhile, GCP (which already had a high proportion of respondents reporting managed Kubernetes use) saw some growth in deploying to serverless Cloud Run from 10% to 17%.
Overall, a majority of respondents were satisfied with using Go on all three major cloud providers, and the figures are statistically unchanged from last year. Respondents reported similar satisfaction levels with Go development for AWS (82% satisfied) and GCP (80%). Azure received a lower satisfaction score (58% satisfied), and free-text responses often cited a need for improvements to Azure's Go SDK and Go support for Azure functions.
The top reasons respondents say they are unable to use Go more remain working on a project in another language (54%), working on a team that prefers to use another language (34%), and the lack of a critical feature in Go itself (26%).
This year we introduced a new option, “I already use Go everywhere I would like to,” so that respondents could opt out of making selections that don't prevent them from using Go. This significantly lowered the rate of selection of all other options, but did not change their relative ordering. We also introduced an option for “Go lacks critical frameworks”.
If we look at only the respondents who selected reasons for not using Go, we can get a better idea of year-over-year trends. Working on an existing project in another language and project/team/lead preference for another language are decreasing over time.
Among the 26% of respondents who said Go lacks language features they need, 88% selected generics as a critical missing feature. Other critical missing features were better error handling (58%), null safety (44%), functional programming features(42%) and a stronger / expanded type system (41%).
To be clear, these numbers are from the subset of respondents who said they would be able to use Go more were it not missing one or more critical features they need, not the entire population of survey respondents. To put that in perspective, 18% of respondents are prevented from using Go because of a lack of generics.
The top challenge respondents reported when using Go was again Go's lack of generics (18%), while modules/package management and problems with learning curve/best practices/docs were both 13%.
This year we asked respondents for their top 5 resources for answering their Go-related questions. Last year we only asked for top 3, so the results aren't directly comparable, however, StackOverflow remains the most popular resource at 65%. Reading source code (57%) remains another popular resource while reliance on godoc.org (39%) has significantly decreased. The package discovery site pkg.go.dev is new to the list this year and was a top resource for 32% of respondents. Respondents who use pkg.go.dev are more likely to agree they are able to quickly find Go packages / libraries they need: 91% for pkg.go.dev users vs. 82% for everyone else.
Over the years, the proportion of respondents who do not attend any Go-related events has been trending up. Due to Covid-19, this year we modified our question around Go events, and found over a quarter of respondents have spent more time in online Go channels than in prior years, and 14% attended a virtual Go meetup, twice as many as last year. 64% of those who attended a virtual event said this was their first virtual event.
We found 12% of respondents identify with a traditionally underrepresented group (e.g., ethnicity, gender identity, et al.), the same as 2019, and 2% identify as women, fewer than in 2019 (3%). Respondents who identified with underrepresented groups showed higher rates of disagreement with the statement "I feel welcome in the Go community" (10% vs. 4%) than those who do not identify with an underrepresented group. These questions allow us to measure diversity in the community and highlight opportunities for outreach and growth.
We added an additional question this year on assistive technology usage, and found that 8% of respondents are using some form of assistive technology. The most commonly used assistive tech was contrast or color settings (2%). This is a great reminder that we have users with accessibility needs and helps drive some of our design decisions on websites managed by the Go team.
The Go team values diversity and inclusion, not simply as the right thing to do, but because diverse voices can illuminate our blindspots and ultimately benefit all users. The way we ask about sensitive information, including gender and traditionally underrepresented groups, has changed according to data privacy regulations and we hope to make these questions, particularly around gender diversity, more inclusive in the future.
Thank you for joining us in reviewing the results of our 2020 developer survey! Understanding developers’ experiences and challenges helps us measure our progress and directs the future of Go. Thanks again to everyone who contributed to this survey—we couldn't have done it without you. We hope to see you next year!
Jean de Klerk, Matt T. Proud 24 February 2021
In many Go APIs, especially modern ones, the first argument to functions and methods is often context.Context. Context provides a means of transmitting deadlines, caller cancellations, and other request-scoped values across API boundaries and between processes. It is often used when a library interacts — directly or transitively — with remote servers, such as databases, APIs, and the like.
context.Context
The documentation for context states:
Contexts should not be stored inside a struct type, but instead passed to each function that needs it.
This article expands on that advice with reasons and examples describing why it's important to pass Context rather than store it in another type. It also highlights a rare case where storing Context in a struct type may make sense, and how to do so safely.
To understand the advice to not store context in structs, let's consider the preferred context-as-argument approach:
type Worker struct { /* … */ } type Work struct { /* … */ } func New() *Worker { return &Worker{} } func (w *Worker) Fetch(ctx context.Context) (*Work, error) { _ = ctx // A per-call ctx is used for cancellation, deadlines, and metadata. } func (w *Worker) Process(ctx context.Context, work *Work) error { _ = ctx // A per-call ctx is used for cancellation, deadlines, and metadata. }
Here, the (*Worker).Fetch and (*Worker).Process methods both accept a context directly. With this pass-as-argument design, users can set per-call deadlines, cancellation, and metadata. And, it's clear how the context.Context passed to each method will be used: there's no expectation that a context.Context passed to one method will be used by any other method. This is because the context is scoped to as small an operation as it needs to be, which greatly increases the utility and clarity of context in this package.
(*Worker).Fetch
(*Worker).Process
context
Let's inspect again the Worker example above with the disfavored context-in-struct approach. The problem with it is that when you store the context in a struct, you obscure lifetime to the callers, or worse intermingle two scopes together in unpredictable ways:
Worker
type Worker struct { ctx context.Context } func New(ctx context.Context) *Worker { return &Worker{ctx: ctx} } func (w *Worker) Fetch() (*Work, error) { _ = w.ctx // A shared w.ctx is used for cancellation, deadlines, and metadata. } func (w *Worker) Process(work *Work) error { _ = w.ctx // A shared w.ctx is used for cancellation, deadlines, and metadata. }
The (*Worker).Fetch and (*Worker).Process method both use a context stored in Worker. This prevents the callers of Fetch and Process (which may themselves have different contexts) from specifying a deadline, requesting cancellation, and attaching metadata on a per-call basis. For example: the user is unable to provide a deadline just for (*Worker).Fetch, or cancel just the (*Worker).Process call. The caller's lifetime is intermingled with a shared context, and the context is scoped to the lifetime where the Worker is created.
The API is also much more confusing to users compared to the pass-as-argument approach. Users might ask themselves:
New
The API would need a good deal of documentation to explicitly tell the user exactly what the context.Context is used for. The user might also have to read code rather than being able to rely on the structure of the API conveys.
And, finally, it can be quite dangerous to design a production-grade server whose requests don't each have a context and thus can't adequately honor cancellation. Without the ability to set per-call deadlines, your process could backlog and exhaust its resources (like memory)!
When Go 1.7 — which introduced context.Context — was released, a large number of APIs had to add context support in backwards compatible ways. For example, net/http's Client methods, like Get and Do, were excellent candidates for context. Each external request sent with these methods would benefit from having the deadline, cancellation, and metadata support that came with context.Context.
net/http
Client
Get
Do
There are two approaches for adding support for context.Context in backwards compatible ways: including a context in a struct, as we'll see in a moment, and duplicating functions, with duplicates accepting context.Context and having Context as their function name suffix. The duplicate approach should be preferred over the context-in-struct, and is further discussed in Keeping your modules compatible. However, in some cases it's impractical: for example, if your API exposes a large number of functions, then duplicating them all might be infeasible.
Context
The net/http package chose the context-in-struct approach, which provides a useful case study. Let's look at net/http's Do. Prior to the introduction of context.Context, Do was defined as follows:
func (c *Client) Do(req *Request) (*Response, error)
After Go 1.7, Do might have looked like the following, if not for the fact that it would break backwards compatibility:
func (c *Client) Do(ctx context.Context, req *Request) (*Response, error)
But, preserving the backwards compatibility and adhering to the Go 1 promise of compatibility is crucial for the standard library. So, instead, the maintainers chose to add a context.Context on the http.Request struct in order to allow support context.Context without breaking backwards compatibility:
http.Request
type Request struct { ctx context.Context // ... } func NewRequestWithContext(ctx context.Context, method, url string, body io.Reader) (*Request, error) { // Simplified for brevity of this article. return &Request{ ctx: ctx, // ... } } func (c *Client) Do(req *Request) (*Response, error)
When retrofitting your API to support context, it may make sense to add a context.Context to a struct, as above. However, remember to first consider duplicating your functions, which allows retrofitting context.Context in a backwards compatibility without sacrificing utility and comprehension. For example:
func (c *Client) Call() error { return c.CallContext(context.Background()) } func (c *Client) CallContext(ctx context.Context) error { // ... }
Context makes it easy to propagate important cross-library and cross-API information down a calling stack. But, it must be used consistently and clearly in order to remain comprehensible, easy to debug, and effective.
When passed as the first argument in a method rather than stored in a struct type, users can take full advantage of its extensibility in order to build a powerful tree of cancelation, deadline, and metadata information through the call stack. And, best of all, its scope is clearly understood when it's passed in as an argument, leading to clear comprehension and debuggability up and down the stack.
When designing an API with context, remember the advice: pass context.Context in as an argument; don't store it in structs.
See the index for more articles.