【解决一个小问题】golang 的 `-race`选项导致 unsafe代码 panic

发布时间 2023-06-13 18:20:25作者: ahfuzhang

作者:张富春(ahfuzhang),转载时请注明作者和引用链接,谢谢!


为了提升性能,使用 unsafe 代码来重构了凯撒加密的代码。代码如下:

const (
	lowerCaseAlphabet = "abcdefghijklmnopqrstuvwxyz"
	upperCaseAlphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
)

var (
	lowerCaseAlphabetArr     = []byte(lowerCaseAlphabet)
	upperCaseAlphabetArr     = []byte(upperCaseAlphabet)
	tableOflowerCaseAlphabet = unsafe.Pointer(&lowerCaseAlphabetArr[0])
	tableOfupperCaseAlphabe  = unsafe.Pointer(&upperCaseAlphabetArr[0])
)

// CaesarFastEncode fast version
func CaesarFastEncode(in []byte, out []byte, rot int) {
	start := unsafe.Pointer(&in[0])
	target := unsafe.Pointer(&out[0])
	for i := 0; i < len(in); i++ {
		c := *((*byte)(start))
		if c == '.' {
			*((*byte)(target)) = '='
		} else if c >= 'a' && c <= 'z' {
			idx := (int(26+(c-'a')) + rot) % 26
			*((*byte)(target)) = *((*byte)(unsafe.Pointer(uintptr(tableOflowerCaseAlphabet) + uintptr(idx))))
		} else if c >= 'A' && c <= 'Z' {
			idx := (int(26+(c-'A')) + rot) % 26
			*((*byte)(target)) = *((*byte)(unsafe.Pointer(uintptr(tableOfupperCaseAlphabe) + uintptr(idx))))
		} else {
			*((*byte)(target)) = *((*byte)(start))
		}
		start = unsafe.Pointer(uintptr(start) + uintptr(1))
		target = unsafe.Pointer(uintptr(target) + uintptr(1))
	}
}

命令行运行go test 的时候发现,代码中发生了 panic。而我直接在 vscode 中通过快捷键运行又是正常的。
错误信息如下:

fatal error: checkptr: pointer arithmetic result points to invalid allocation

goroutine 6 [running]:
runtime.throw({0x104936d70?, 0x10410270c?})
        /opt/homebrew/Cellar/go/1.20.4/libexec/src/runtime/panic.go:1047 +0x40 fp=0xc0001e5a60 sp=0xc0001e5a30 pc=0x104132280
runtime.checkptrArithmetic(0xc0001e5ac8?, {0xc0001e5b00, 0x1, 0x0?})
        /opt/homebrew/Cellar/go/1.20.4/libexec/src/runtime/checkptr.go:69 +0xac fp=0xc0001e5a90 sp=0xc0001e5a60 pc=0x1041028cc
cryptoutil.CaesarFastEncode({0xc000216cc4, 0xc, 0xc0001e5b68?}, {0xc000232a02, 0x1fe, 0x104f3ee00?}, 0x3)
        /Users/ahfuzhang/code/golang/xxx/caesar.go:83 +0x84 fp=0xc0001e5b20 sp=0xc0001e5a90 pc=0x104570a74

进一步发现,命令行中有个不一样的选项:

go test -v -cover -race ./...

去掉 -race选项后,一切正常。

搜索看到了这篇文章:《Go 1.15中值得关注的几个变化

Go 1.14版本中,Go编译器在被传入-race和-msan的情况下,默认会执行-d=checkptr,即对unsafe.Pointer的使用进行合法性检查。-d=checkptr主要检查两项内容:

•当将unsafe.Pointer转型为*T时,T的内存对齐系数不能高于原地址的;

•做完指针算术后,转换后的unsafe.Pointer仍应指向原先Go堆对象

由此可见,循环做完后,最后一行必然导致指针超出原来的 buffer。
为了符合 golang 的规范,微调了代码后通过:

// CaesarFastEncode fast version
func CaesarFastEncode(in []byte, out []byte, rot int) {
	start := unsafe.Pointer(&in[0])
	end := uintptr(start) + uintptr(len(in)-1)
	target := (unsafe.Pointer(&out[0]))
	for {
		c := *((*byte)(start))
		if c == '.' {
			*((*byte)(target)) = '='
		} else if c >= 'a' && c <= 'z' {
			idx := (int(26+(c-'a')) + rot) % 26
			*((*byte)(target)) = *((*byte)(unsafe.Pointer(uintptr(tableOflowerCaseAlphabet) + uintptr(idx))))
		} else if c >= 'A' && c <= 'Z' {
			idx := (int(26+(c-'A')) + rot) % 26
			*((*byte)(target)) = *((*byte)(unsafe.Pointer(uintptr(tableOfupperCaseAlphabe) + uintptr(idx))))
		} else {
			*((*byte)(target)) = *((*byte)(unsafe.Pointer(start)))
		}
		if uintptr(start) >= end {
			break
		}
		start = unsafe.Pointer(uintptr(start) + uintptr(1))
		target = unsafe.Pointer(uintptr(target) + uintptr(1))
	}
}

由此看来,只要使用了 unsafe 代码,都应该加上-race选项。