I want off Mr. Golang’s Wild Ride (2020)
These modules are quite nice!!
Feb 28, 2020
· 31 minute read
·
golang
·
rant
My honeymoon with the Go language is extremely over.
This article is going to have a different tone from what I’ve been posting
the past year – it’s a proper rant. And I always feel bad writing those,
because, inevitably, it discusses things a lot of people have been working
very hard on.
In spite of that, here we are.
Having invested thousands of hours into the language, and implemented several
critical (to my employer) pieces of infrastructure with it, I wish I hadn’t.
If you’re already heavily invested in Go, you probably shouldn’t read this,
it’ll probably just twist the knife. If you work on Go, you definitely
shouldn’t read this.
I’ve been suffering Go’s idiosyncracies in relative silence for too long,
there’s a few things I really need to get off my chest.
Alright? Alright.
Garden-variety takes on Go
By now, everybody knows Go doesn’t have generics, which makes a lot of
problems impossible to model accurately (instead, you have to fall back to
reflection, which is extremely unsafe, and the API is very error-prone),
error handling is wonky (even with your pick of the third-party libraries
that add context or stack traces), package management took a while to arrive,
etc.
But everybody also knows Go’s strengths: static linking makes binaries easy
to deploy (although, Go binaries get very
large,
even if you strip DWARF tables – stack trace annotations still remain, and are costly).
Compile times are short (unless you need cgo), there’s an interactive runtime
profiler (pprof) at arm’s reach, it’s relatively cross-platform (there’s even
a tiny variant for embedded), it’s
easy to syntax-highlight, and there’s now an official LSP
server for it.
I’ve accepted all of these – the good and the bad.
We’re here to talk about the ugly.
Simple is a lie
Over and over, every piece of documentation for the Go language markets it
as “simple”.
This is a lie.
Or rather, it’s a half-truth that conveniently covers up the fact that, when
you make something simple, you move complexity elsewhere.
Computers, operating systems, networks are a hot mess. They’re barely
manageable, even if you know a decent amount about what you’re doing. Nine
out of ten software engineers agree: it’s a miracle anything works at all.
So all the complexity is swept under the rug. Hidden from view, but not
solved.
Here’s a simple example.
This example does go on for a while, actually – but don’t let the specifics
distract you. While it goes rather in-depth, it illustrates a larger point.
Most of Go’s APIs (much like NodeJS’s APIs) are designed for Unix-like
operating systems. This is not surprising, as Rob & Ken are from the Plan 9
gang.
So, the file API in Go looks like this:
Go code
// File represents an open file descriptor.
type File struct {
*file // os specific
}
func (f *File) Stat() (FileInfo, error) {
// omitted
}
// A FileInfo describes a file and is returned by Stat and Lstat.
type FileInfo interface {
Name() string // base name of the file
Size() int64 // length in bytes for regular files; system-dependent for others
Mode() FileMode // file mode bits
ModTime() time.Time // modification time
IsDir() bool // abbreviation for Mode().IsDir()
Sys() interface{} // underlying data source (can return nil)
}
// A FileMode represents a file’s mode and permission bits.
// The bits have the same definition on all systems, so that
// information about files can be moved from one system
// to another portably. Not all bits apply to all systems.
// The only required bit is ModeDir for directories.
type FileMode uint32
// The defined file mode bits are the most significant bits of the FileMode.
// The nine least-significant bits are the standard Unix rwxrwxrwx permissions.
// The values of these bits should be considered part of the public API and
// may be used in wire protocols or disk representations: they must not be
// changed, although new bits might be added.
const (
// The single letters are the abbreviations
// used by the String method’s formatting.
ModeDir FileMode=1 /i> (32 – 1 – iota) // d: is a directory
ModeAppend // a: append-only
ModeExclusive // l: exclusive use
ModeTemporary // T: temporary file; Plan 9 only
ModeSymlink // L: symbolic link
ModeDevice // D: device file
ModeNamedPipe // p: named pipe (FIFO)
ModeSocket // S: Unix domain socket
ModeSetuid // u: setuid
ModeSetgid // g: setgid
ModeCharDevice // c: Unix character device, when ModeDevice is set
ModeSticky // t: sticky
ModeIrregular // ?: non-regular file; nothing else is known about this file
// Mask for the type bits. For regular files, none will be set.
ModeType=ModeDir | ModeSymlink | ModeNamedPipe | ModeSocket | ModeDevice | ModeCharDevice | ModeIrregular
ModePerm FileMode=0777 // Unix permission bits
)
Makes sense for a Unix, right?
Every file has a mode, there’s even a command that lets you dump it as hex:
Shell session
$ stat -c ‘%f’ /etc/hosts
81a4
$ stat -c ‘%f’ /usr/bin/man
81ed
And so, a simple Go program can easily grab those “Unix permission bits”:
Go code
package main
import (
“fmt”
“os”
)
func main() {
arg :=os.Args[1]
fi, _ :=os.Stat(arg)
fmt.Printf(“(%s) mode=%on”, arg, fi.Mode() & os.ModePerm)
}
Shell session
$ go run main.go /etc/hosts
(/etc/hosts) mode=644
$ go run main.go /usr/bin/man
(/etc/hosts) mode=755
On Windows, files don’t have modes. It doesn’t have stat, lstat, fstat
syscalls – it has a FindFirstFile family of functions (alternatively,
CreateFile to open, then GetFileAttributes, alternatively,
GetFileInformationByHandle), which takes a pointer to a WIN32_FIND_DATA
structure, which contains file
attributes.
So, what happens if you run that program on Windows?
Shell session
> go run main.go C:Windowsnotepad.exe
(C:Windowsnotepad.exe) mode=666
It makes up a mode.
Go code
// src/os/types_windows.go
func (fs *fileStat) Mode() (m FileMode) {
if fs==&devNullStat {
return ModeDevice | ModeCharDevice | 0666
}
if fs.FileAttributes&syscall.FILE_ATTRIBUTE_READONLY !=0 {
m |=0444
} else {
m |=0666
}
if fs.isSymlink() {
return m | ModeSymlink
}
if fs.FileAttributes&syscall.FILE_ATTRIBUTE_DIRECTORY !=0 {
m |=ModeDir | 0111
}
switch fs.filetype {
case syscall.FILE_TYPE_PIPE:
m |=ModeNamedPipe
case syscall.FILE_TYPE_CHAR:
m |=ModeDevice | ModeCharDevice
}
return m
}
Node.js does the same. There’s a single fs.Stats
“type” for all platforms.
Using “whatever Unix has” as the lowest common denominator is extremely
common in open-source codebases, so it’s not surprising.
Let’s go a little bit further. On Unix systems, you can change the modes of
files, to make them read-only, or flip the executable bit.
Go code
package main
import (
“fmt”
“os”
)
func main() {
arg :=os.Args[1]
fi, err :=os.Stat(arg)
must(err)
fmt.Printf(“(%s) old mode=%on”, arg, fi.Mode()&os.ModePerm)
must(os.Chmod(arg, 0755))
fi, err=os.Stat(arg)
must(err)
fmt.Printf(“(%s) new mode=%on”, arg, fi.Mode()&os.ModePerm)
}
func must(err error) {
if err !=nil {
panic(err)
}
}
Let’s run this on Linux:
Shell session
$ touch test.txt
$ go run main.go test.txt
(test.txt) old mode=644
(test.txt) new mode=755
And now on Windows:
Shell session
> go run main.go test.txt
(test.txt) old mode=666
(test.txt) new mode=666
So, no errors. Chmod just silently does… nothing. Which is reasonable –
there’s no equivalent to the “executable bit” for files on Windows.
What does Chmod even do on Windows?
Go code
// src/syscall/syscall_windows.go
func Chmod(path string, mode uint32) (err error) {
p, e :=UTF16PtrFromString(path)
if e !=nil {
return e
}
attrs, e :=GetFileAttributes(p)
if e !=nil {
return e
}
if mode&S_IWRITE !=0 {
attrs &^=FILE_ATTRIBUTE_READONLY
} else {
attrs |=FILE_ATTRIBUTE_READONLY
}
return SetFileAttributes(p, attrs)
}
It sets or clears the read-only bit. That’s it.
We have an uint32 argument, with four billion two hundred ninety-four
million nine hundred sixty-seven thousand two hundred ninety-five possible
values, to encode… one bit of information.
That’s a pretty innocent lie. The assumption that files have modes was baked
into the API design from the start, and now, everyone has to live with it.
Just like in Node.JS, and probably tons of other languages.
But it doesn’t have to be like that.
A language with a more involved type system, and better designed libraries
could avoid that pitfall.
Out of curiosity, what does Rust do?
Oh, here we go again – Rust, Rust, and Rust again.
Why always Rust?
Well, I tried real hard to keep Rust out of all of this. Among other
things, because people are going to dismiss this article as coming from “a
typical rustacean”.
But for all the problems I raise in this article… Rust gets it right.
If I had another good example, I’d use it. But I don’t, so, here goes.
There’s no stat-like function in the Rust standard library. There’s
std::fs::metadata:
Rust code
pub fn metadata(path: P) -> Result
This function signatures tells us a lot already. It returns a Result, which
means, not only do we know this can fail, we have to handle it. Either by
panicking on error, with .unwrap() or .expect(), or by matching it against
Result::Ok / Result::Err, or by bubbling it up with the ? operator.
The point is, this function signature makes it impossible for us to access
an invalid/uninitialized/null Metadata. With a Go function, if you ignore the
returned error, you still get the result – most probably a null pointer.
Also, the argument is not a string – it’s a path. Or rather, it’s something
that can be turned into a path.
And String does implement
AsRef,
so, for simple use cases, it’s not troublesome:
Rust code
fn main() {
let metadata=std::fs::metadata(“Cargo.toml”).unwrap();
println!(“is dir? {:?}”, metadata.is_dir());
println!(“is file? {:?}”, metadata.is_file());
}
But paths are not necessarily strings. On Unix (!), paths can be any sequence
of bytes, except null bytes.
Shell session
$ cd rustfun/
$ touch “$(printf “xbdxb2x3dxbcx20xe2x8cx98″)”
$ ls
ls: cannot compare file names ‘Cargo.lock’ and ‘275262=274 ⌘’: Invalid or incomplete multibyte or wide character
src target Cargo.lock Cargo.toml ”$’275262”=’$’274” ⌘’
We’ve just made a file with a very naughty name – but it’s a perfectly
valid file, even if ls struggles with it.
Shell session
$ stat “$(printf “xbdxb2x3dxbcx20xe2x8cx98″)”
File:=⌘
Size: 0 Blocks: 0 IO Block: 65536 regular empty file
Device: 8c70d496h/2356204694d Inode: 72620543991375285 Links: 1
Access: (0644/-rw-r–r–) Uid: (197611/ amos) Gid: (197611/ amos)
Access: 2020-02-28 13:12:12.783734000 +0100
Modify: 2020-02-28 13:12:12.783734000 +0100
Change: 2020-02-28 13:12:12.783329400 +0100
Birth: 2020-02-28 13:12:12.783329400 +0100
That’s not something we can represent with a String in Rust, because Rust
Strings are valid utf-8, and this isn’t.
Rust Paths, however, are… arbitrary byte sequences.
And so, if we use std::fs::read_dir, we have no problem listing it and
getting its metadata:
Rust code
use std::fs;
fn main() {
let entries=fs::read_dir(“.”).unwrap();
for entry in entries {
let path=entry.unwrap().path();
let meta=fs::metadata(&path).unwrap();
if meta.is_dir() {
println!(“(dir) {:?}”, path);
} else {
println!(” {:?}”, path);
}
}
}
Shell session
$ cargo run –quiet
(dir) “./src”
“./Cargo.toml”
“./.gitignore”
“./xBDxB2=xBC ⌘”
(dir) “./.git”
“./Cargo.lock”
(dir) “./target”
What about Go?
Go code
package main
import (
“fmt”
“os”
)
func main() {
arg :=os.Args[1]
f, err :=os.Open(arg)
must(err)
entries, err :=f.Readdir(-1)
must(err)
for _, e :=range entries {
if e.IsDir() {
fmt.Printf(“(dir) %sn”, e.Name())
} else {
fmt.Printf(” %sn”, e.Name())
}
}
}
func must(err error) {
if err !=nil {
panic(err)
}
}
Shell session
$ go build
$ ./gofun ../rustfun
(dir) src
Cargo.toml
.gitignore
=⌘
(dir) .git
Cargo.lock
(dir) target
It… silently prints a wrong version of the path.
See, there’s no “path” type in Go. Just “string”. And Go strings are just byte
slices, with no guarantees about what’s
inside.
So it prints garbage, whereas in Rust, Path does not implement Display, so
we couldn’t do this:
Rust code
println!(“(dir) {}”, path);
We had to do this:
Rust code
println!(“(dir) {:?}”, path);
And if we wanted a friendlier output, we could handle both cases: when
the path happens to be a valid utf-8 string, and when it doesn’t:
Rust code
use std::fs;
fn main() {
let entries=fs::read_dir(“.”).unwrap();
for entry in entries {
let path=entry.unwrap().path();
let meta=fs::metadata(&path).unwrap();
let prefix=if meta.is_dir() {
“(dir)”
} else {
” ”
};
match path.to_str() {
Some(s)=> println!(“{} {}”, prefix, s),
None=> println!(“{} {:?} (invalid utf-8)”, prefix, path),
}
}
}
Shell session
$ cargo run –quiet
(dir) ./src
./Cargo.toml
./.gitignore
“./xBDxB2=xBC ⌘” (invalid utf-8)
(dir) ./.git
./Cargo.lock
(dir) ./target
Go says “don’t worry about encodings! things are probably utf-8”.
Except when they aren’t. And paths aren’t. So, in Go, all path manipulation
routines operate on string, let’s take a look at the path/filepath package.
Package filepath implements utility routines for manipulating filename paths
in a way compatible with the target operating system-defined file paths.
The filepath package uses either forward slashes or backslashes, depending on
the operating system. To process paths such as URLs that always use forward
slashes regardless of the operating system, see the path package.
What does this package give us?
Go code
func Abs(path string) (string, error)
func Base(path string) string
func Clean(path string) string
func Dir(path string) string
func EvalSymlinks(path string) (string, error)
func Ext(path string) string
func FromSlash(path string) string
func Glob(pattern string) (matches []string, err error)
func HasPrefix(p, prefix string) bool
func IsAbs(path string) bool
func Join(elem …string) string
func Match(pattern, name string) (matched bool, err error)
func Rel(basepath, targpath string) (string, error)
func Split(path string) (dir, file string)
func SplitList(path string) []string
func ToSlash(path string) string
func VolumeName(path string) string
func Walk(root string, walkFn WalkFunc) error
Strings. Lots and lots of strings. Well, byte slices.
Speaking of bad design decisions – what’s that Ext function I see?
Go code
// Ext returns the file name extension used by path. The extension is the suffix
// beginning at the final dot in the final element of path; it is empty if there
// is no dot.
func Ext(path string) string
Interesting! Let’s try it out.
Go code
package main
import (
“fmt”
“path/filepath”
)
func main() {
inputs :=[]string{
“/”,
“/.”,
“/.foo”,
“/foo”,
“/foo.txt”,
“/foo.txt/bar”,
“C:\”,
“C:\.”,
“C:\foo.txt”,
“C:\foo.txt\bar”,
}
for _, i :=range inputs {
fmt.Printf(“%24q=> %qn”, i, filepath.Ext(i))
}
}
func must(err error) {
if err !=nil {
panic(err)
}
}
Shell session
$ go run main.go
“/”=> “”
“/.”=> “.”
“/.foo”=> “.foo”
“/foo”=> “”
“/foo.txt”=> “.txt”
“/foo.txt/bar”=> “”
“C:\”=> “”
“C:\.”=> “.”
“C:\foo.txt”=> “.txt”
“C:\foo.txt\bar”=> “.txt\bar”
Right away, I’m in debating mood – is .foo’s extension really .foo? But
let’s move on.
This example was run on Linux, so C:foo.txtbar’s extension, according
to filepath.Ext, is.. .txtbar.
Why? Because the Go standard library makes the assumption that a platform has
a single path separator – on Unix and BSD-likes, it’s /, and on Windows
it’s \.
Except… that’s not the whole truth. I was curious, so I checked:
C code
// in `fun.c`
void main() {
HANDLE hFile=CreateFile(“C:/Users/amos/test.txt”, GENERIC_WRITE, 0, NULL,
CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL);
char *data=”Hello from the Win32 API”;
DWORD dwToWrite=(DWORD) strlen(data);
DWORD dwWritten=0;
WriteFile(hFile, data, dwToWrite, &dwWritten, NULL);
CloseHandle(hFile);
}
Shell session
> cl fun.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.23.28107 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
fun.c
Microsoft (R) Incremental Linker Version 14.23.28107.0
Copyright (C) Microsoft Corporation. All rights reserved.
/out:fun.exe
fun.obj
> .fun.exe
> type C:Usersamostest.txt
Hello from the Win32 API
No funny Unix emulation business going on – just
regular old Windows 10.
And yet, in Go’s standard library, the path/filepath package exports those
constants:
Go code
const (
Separator =os.PathSeparator
ListSeparator=os.PathListSeparator
)
os, in turn, exports:
Go code
// src/os/path_windows.go
const (
PathSeparator =’\’ // OS-specific path separator
PathListSeparator=’;’ // OS-specific path list separator
)
So how comes filepath.Ext works with both separators on Windows?
Shell session
$ go run main.go
“/”=> “”
“/.”=> “.”
“/.foo”=> “.foo”
“/foo”=> “”
“/foo.txt”=> “.txt”
“/foo.txt/bar”=> “”
“C:\”=> “”
“C:\.”=> “.”
“C:\foo.txt”=> “.txt”
“C:\foo.txt\bar”=> “”
Let’s look at its implementation:
Go code
// src/path/filepath/path.go
func Ext(path string) string {
for i :=len(path) – 1; i>=0 && !os.IsPathSeparator(path[i]); i– {
if path[i]==’.’ {
return path[i:]
}
}
return “”
}
Ah. An IsPathSeparator function.
Sure enough:
Go code
// src/os/path_windows.go
// IsPathSeparator reports whether c is a directory separator character.
func IsPathSeparator(c uint8) bool {
// NOTE: Windows accept / as path separator.
return c==’\’ || c==’/’
}
(Can I just point out how hilarious that “Extension” was deemed long
enough to abbreviate to “Ext”, but “IsPathSeparator” wasn’t?)
How does Rust handle this?
It has std::path::is_separator:
Rust code
/// Determines whether the character is one of the permitted
// path separators for the current platform.
pub fn is_separator(c: char) -> bool
And it has std::path::MAIN_SEPARATOR – emphasis on main separator:
Rust code
/// The primary separator of path components for the current platform.
///
/// For example, / on Unix and on Windows.
pub const MAIN_SEPARATOR: char
The naming along makes it much clearer that there might be secondary path
separators, and the rich Path manipulation API makes it much less likely
to find this kind of code, for example:
Go code
DefaultScripts=”downloads” + string(os.PathSeparator) + “defaultScripts”
Or this kind:
Go code
if os.PathSeparator==’/’ {
projname=strings.Replace(name, “\”, “/”, -1)
} else if os.PathSeparator==’\’ {
projname=strings.Replace(name, “/”, “\”, -1)
}
Or this… kind:
Go code
filefullpath=fmt.Sprintf(“%s%c%s%c%s%c%s%c%s%s”,
a.DataDir, os.PathSeparator,
m[0:1], os.PathSeparator,
m[1:2], os.PathSeparator,
m[2:3], os.PathSeparator,
m, ext)
It turns out Rust also has a “get a path’s extension” function, but it’s a lot
more conservative in the promises it makes:
Rust code
// Extracts the extension of self.file_name, if possible.
//
// The extension is:
//
// * None, if there is no file name;
// * None, if there is no embedded .;
// * None, if the file name begins with . and has no other .s within;
// * Otherwise, the portion of the file name after the final .
pub fn extension(&self) -> Option
Let’s submit it to the same test:
Rust code
fn main() {
let inputs=[
r”/”,
r”/.”,
r”/.foo”,
r”/foo”,
r”/foo.txt”,
r”/foo.txt/bar”,
r”C:”,
r”C:.”,
r”C:foo.txt”,
r”C:foo.txtbar”,
];
for input in &inputs {
use std::path::Path;
println!(“{:>20}=> {:?}”, input, Path::new(input).extension());
}
}
On Linux:
Shell session
$ cargo run –quiet
/=> None
/.=> None
/.foo=> None
/foo.=> Some(“”)
/foo=> None
/foo.txt=> Some(“txt”)
/foo.txt/bar=> None
C:=> None
C:.=> Some(“”)
C:foo.txt=> Some(“txt”)
C:foo.txtbar=> Some(“txt\bar”)
On Windows:
Shell session
$ cargo run –quiet
/=> None
/.=> None
/.foo=> None
/foo.=> Some(“”)
/foo=> None
/foo.txt=> Some(“txt”)
/foo.txt/bar=> None
C:=> None
C:.=> None
C:foo.txt=> Some(“txt”)
C:foo.txtbar=> None
Like Go, it gives a txtbar extension for a Windows path on Linux.
Unlike Go, it:
Doesn’t think “/.foo” has a file extension
Distinguishes between the “/foo.” case (Some(“”)) and the “/foo” case (None)
Let’s also look at the Rust implementation of std::path::Path::extension:
Rust code
pub fn extension(&self) -> Option {
self.file_name().map(split_file_at_dot).and_then(|(before, after)| before.and(after))
}
Let’s dissect that: first it calls file_name(). How does that work? Is it where
it searches for path separators backwards from the end of the path?
Rust code
pub fn file_name(&self) -> Option {
self.components().next_back().and_then(|p| match p {
Component::Normal(p)=> Some(p.as_ref()),
_=> None,
})
}
No! It calls components which returns a type that implements DoubleEndedIterator –
an iterator you can navigate from the front or the back. Then it grabs the first item
from the back – if any – and returns that.
The iterator does look for path separators – lazily, in a re-usable way. There is
no code duplication, like in the Go library:
Go code
// src/os/path_windows.go
func dirname(path string) string {
vol :=volumeName(path)
i :=len(path) – 1
for i>=len(vol) && !IsPathSeparator(path[i]) {
i–
}
dir :=path[len(vol) : i+1]
last :=len(dir) – 1
if last> 0 && IsPathSeparator(dir[last]) {
dir=dir[:last]
}
if dir==”” {
dir=”.”
}
return vol + dir
}
So, now we have only the file name. If we had /foo/bar/baz.txt, we’re now only
dealing with baz.txt – as an OsStr, not a utf-8 String. We can still have
random bytes.
We then map this result through split_file_at_dot, which behaves like so:
For “foo”, return (Some(“foo”), None)
For “foo.bar”, return (Some(“foo”), Some(“bar”))
For “foo.bar.baz”, return (Some(“foo.bar”), Some(“baz”))
and_then, we only return after if before wasn’t None.
If we spelled out everything, we’d have:
Rust code
pub fn extension(&self) -> Option {
if let Some(file_name)=self.file_name() {
let (before, after)=split_file_at_dot(file_name);
if let Some(before) {
// note: `after` is already an `Option` – it
// might still be `None`.
return after
}
}
None
}
The problem is carefully modelled. We can look at what we’re manipulating
just by looking at its type. If it might not exist, it’s an Option! If
it’s a path with multiple components, it’s a &Path (or its owned
counterpart, PathBuf). If it’s just part of a path, it’s an &OsStr.
Of course there’s a learning curve. Of course there’s more concepts involved
than just throwing for loops at byte slices and seeing what sticks, like the
Go library does.
But the result is a high-performance, reliable and type-safe library.
It’s worth it.
Speaking of Rust, we haven’t seen how it handles the whole “mode” thing yet.
So std::fs::Metadata has is_dir() and is_file(), which return booleans.
It also has len(), which returns an u64 (unsigned 64-bit integer).
It has created(), modified(), and accessed(), all of which return an
Option. Again – the types inform us on what scenarios are
possible. Access timestamps might not exist at all.
The returned time is not an std::time::Instant – it’s an
std::time::SystemTime – the documentation tells us the difference:
A measurement of the system clock, useful for talking to external entities
like the file system or other processes.
Distinct from the
Instant type,
this time measurement is not monotonic. This means that you can save a
file to the file system, then save another file to the file system, and the
second file has a SystemTime measurement earlier than the first. In other
words, an operation that happens after another operation in real time may
have an earlier SystemTime!
Consequently, comparing two SystemTime instances to learn about the
duration between them returns a
Result instead of
an infallible
Duration to
indicate that this sort of time drift may happen and needs to be handled.
Although a SystemTime cannot be directly inspected, the
UNIX_EPOCH
constant is provided in this module as an anchor in time to learn information
about a SystemTime. By calculating the duration from this fixed point in
time, a SystemTime can be converted to a human-readable time, or perhaps
some other string representation.
The size of a SystemTime struct may vary depending on the target operating
system.
Source: https://doc.rust-lang.org/std/time/struct.SystemTime.html
What about permissions? Well, there it is:
Rust code
pub fn permissions(&self) -> Permissions
A Permissions type! Just for that! And we can afford it, too – because
types don’t cost anything at runtime. Everything probably ends up inlined
anyway.
What does it expose?
Rust code
pub fn readonly(&self) -> bool {}
pub fn set_readonly(&mut self, readonly: bool) {}
Well! It exposes only what all supported operating systems have in common.
Can we still get Unix permission? Of course! But only on Unix:
Representation of the various permissions on a file.
This module only currently provides one bit of information,
readonly,
which is exposed on all currently supported platforms. Unix-specific
functionality, such as mode bits, is available through the
PermissionsExt
trait.
Source: https://doc.rust-lang.org/std/fs/struct.Permissions.html
std::os::unix::fs::PermissionsExt is only compiled in on Unix, and exposes
the following functions:
Rust code
fn mode(&self) -> u32 {}
fn set_mode(&mut self, mode: u32) {}
fn from_mode(mode: u32) -> Self {}
The documentation makes it really clear it’s Unix-only:
But it’s not just documentation. This sample program will compile and run
on Linux (and macOS, etc.)
Rust code
use std::fs::File;
use std::os::unix::fs::PermissionsExt;
fn main() -> std::io::Result {
let f=File::open(“/usr/bin/man”)?;
let metadata=f.metadata()?;
let permissions=metadata.permissions();
println!(“permissions: {:o}”, permissions.mode());
Ok(())
}
Shell session
$ cargo run –quiet
permissions: 100755
But will fail to compile on Windows:
Shell session
$ cargo run –quiet
error[E0433]: failed to resolve: could not find `unix` in `os`
–> srcmain.rs:2:14
|
2 | use std::os::unix::fs::PermissionsExt;
| ^^^^ could not find `unix` in `os`
error[E0599]: no method named `mode` found for type `std::fs::Permissions` in the current scope
–> srcmain.rs:9:47
|
9 | println!(“permissions: {:o}”, permissions.mode());
| ^^^^ method not found in `std::fs::Permissions`
error: aborting due to 2 previous errors
Some errors have detailed explanations: E0433, E0599.
For more information about an error, try `rustc –explain E0433`.
error: could not compile `rustfun`.
To learn more, run the command again with –verbose.
How can we make a program that runs on Windows too? The same way
the standard library only exposes PermissionsExt on Unix: with
attributes.
Rust code
use std::fs::File;
#[cfg(target_family=”unix”)]
use std::os::unix::fs::PermissionsExt;
fn main() -> std::io::Result {
let arg=std::env::args().nth(1).unwrap();
let f=File::open(&arg)?;
let metadata=f.metadata()?;
let permissions=metadata.permissions();
#[cfg(target_family=”unix”)]
{
println!(“permissions: {:o}”, permissions.mode());
}
#[cfg(target_family=”windows”)]
{
println!(“readonly? {:?}”, permissions.readonly());
}
Ok(())
}
Those aren’t #ifdef – they’re not preprocessor directives. There’s no risk
of forgetting an #endif. And if you miss if/else chains, there’s a crate
for that.
Here’s that sample program on Linux:
Shell session
$ cargo run –quiet — /usr/bin/man
permissions: 100755
And on Windows:
Shell session
$ cargo run –quiet — Cargo.toml
readonly? false
Can you do that in Go? Sure! Kind of!
There’s two ways to do something similar, and both involve multiple files.
Here’s one:
Shell session
$ go mod init github.com/fasterthanlime/gofun
In main.go, we need:
Go code
package main
import “os”
func main() {
poke(os.Args[1])
}
In poke_windows.go, we need:
Go code
package main
import (
“fmt”
“os”
)
func poke(path string) {
stats, _ :=os.Stat(path)
fmt.Printf(“readonly? %vn”, (stats.Mode() & 0o600)==0);
}
And in poke_unix.go, we need:
Go code
// +build !windows
package main
import (
“fmt”
“os”
)
func poke(path string) {
stats, _ :=os.Stat(path)
fmt.Printf(“permissions: %on”, stats.Mode() & os.ModePerm);
}
Note how the _windows.go suffix is magic – it’ll get automatically excluded
on non-Windows platforms. There’s no magic suffix for Unix systems though!
So we have to add a build constraint, which is:
A comment
That must be “near the top of the file”
That can only be preceded by blank space
That must appear before the package clause
That has its own language
From the docs:
A build constraint is evaluated as the OR of space-separated options. Each
option evaluates as the AND of its comma-separated terms. Each term consists
of letters, digits, underscores, and dots. A term may be negated with a
preceding !. For example, the build constraint:
// +build linux,386 darwin,!cgo
corresponds to the boolean formula:
(linux AND 386) OR (darwin AND (NOT cgo))
A file may have multiple build constraints. The overall constraint is the AND
of the individual constraints. That is, the build constraints:
// +build linux darwin
// +build 386
corresponds to the boolean formula:
(linux OR darwin) AND 386
Fun! Fun fun fun. So, on Linux, we get:
Shell session
$ go build
$ ./gofun /usr/bin/man
permissions: 755
$ ./gofun /etc/hosts
permissions: 644
And on Windows, we get:
Shell session
> go build
> .gofun.exe .main.go
readonly? false
Now, at least there’s a way to write platform-specific code in Go.
In practice, it gets old very quickly. You now have related code split across
multiple files, even if only one of the functions is platform-specific.
Build constraints override the magic suffixes, so it’s never obvious exactly
which files are compiled in. You also have to duplicate (and keep in sync!)
function signatures all over the place.
It’s… a hack. A shortcut. And an annoying one, at that.
So what happens when you make it hard for users to do things the right way?
(The right way being, in this case, to not compile in code that isn’t relevant
for a given platform). They take shortcuts, too.
Even in the official Go distribution, a lot of code just switches on the value
of runtime.GOOS at, well, run-time:
Go code
// src/net/file_test.go
func TestFileConn(t *testing.T) {
switch runtime.GOOS {
case “plan9”, “windows”:
t.Skipf(“not supported on %s”, runtime.GOOS)
}
for _, tt :=range fileConnTests {
if !testableNetwork(tt.network) {
t.Logf(“skipping %s test”, tt.network)
continue
}
“But these are little things!”
They’re all little things. They add up. Quickly.
And they’re symptomatic of the problems with “the Go way” in general. The Go
way is to half-ass things.
The Go way is to patch things up until they sorta kinda work, in the name of
simplicity.
Lots of little things
Speaking of little things, let’s consider what pushed me over the edge and
provoked me to write this whole rant in the first place.
It was this package.
What does it do?
Provides mechanisms for adding idle timeouts to net.Conn and net.Listener.
Why do we need it?
Because the real-world is messy.
If you do a naive HTTP request in Go:
Go code
package main
import (
“fmt”
“io/ioutil”
“net/http”
)
func main() {
res, err :=http.Get(“http://perdu.com”)
must(err)
defer res.Body.Close() // this is a *very* common gotcha
body, err :=ioutil.ReadAll(res.Body)
must(err)
fmt.Printf(“%s”, string(body))
}
func must(err error) {
if err !=nil {
panic(err)
}
}
Shell session
$ go run main.go
Perdu sur l’Internet ?Pas de panique, on va vous aider *
Read More
Share this on knowasiak.com to discuss with people on this topicSign up on Knowasiak.com now if you’re not registered yet.
Responses