Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sentry: catch.go:29: runtime error: invalid memory address or nil pointer dereference (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/u... #124338

Closed
cockroach-sentry opened this issue May 17, 2024 · 1 comment
Closed

Sentry: catch.go:29: runtime error: invalid memory address or nil pointer dereference (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/u... #124338

cockroach-sentry opened this issue May 17, 2024 · 1 comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented May 17, 2024 •
edited by cockroach-jira-scripts

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/5363726734/?referrer=webhooks_plugin

Panic Message:

catch.go:29: runtime error: invalid memory address or nil pointer dereference
(1) assertion failure
Wraps: (2) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/util/errorutil.ShouldCatch
  | 	github.com/cockroachdb/cockroach/pkg/util/errorutil/catch.go:29
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).Optimize.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:248
  | runtime.gopanic
  | 	GOROOT/src/runtime/panic.go:884
  | runtime.panicmem
  | 	GOROOT/src/runtime/panic.go:260
  | runtime.sigpanic
  | 	GOROOT/src/runtime/signal_unix.go:839
  | github.com/cockroachdb/cockroach/pkg/sql/opt.(*Metadata).TableAnnotation
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/table_meta.go:552
  | github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*statisticsBuilder).makeTableStatistics
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/memo/statistics_builder.go:589
  | github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*statisticsBuilder).availabilityFromInput
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/memo/statistics_builder.go:283
  | github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*statisticsBuilder).buildScan
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/memo/statistics_builder.go:793
  | github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*logicalPropsBuilder).buildScanProps
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/memo/logical_props_builder.go:186
  | github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*Memo).MemoizeScan
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/opt/memo/expr.og.go:19990
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*CustomFuncs).generateLookupJoinsImpl
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/join_funcs.go:400
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*CustomFuncs).GenerateLookupJoins
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/join_funcs.go:261
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*explorer).exploreInnerJoin
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/opt/xform/explorer.og.go:830
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*explorer).exploreGroupMember
  | 	github.com/cockroachdb/cockroach/bazel-out/k8-opt/bin/pkg/sql/opt/xform/explorer.og.go:28
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*explorer).exploreGroup
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/explorer.go:185
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroup
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:536
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:299
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroupMember
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:578
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroup
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:523
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:299
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroupMember
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:578
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroup
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:523
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:299
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroupMember
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:578
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeGroup
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:523
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:299
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeScalarExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:621
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:311
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeScalarExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:621
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:311
  | github.com/cockroachdb/cockroach/pkg/sql/opt/xform.(*Optimizer).optimizeScalarExpr
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/xform/optimizer.go:621
Wraps: (3) runtime error: invalid memory address or nil pointer dereference
  | -- cause hidden behind barrier
  | runtime error: invalid memory address or nil pointer dereference
  | (1) runtime error: invalid memory address or nil pointer dereference
  | Error types: (1) runtime.errorString
Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *barriers.barrierErr
-- report composition:
*barriers.barrierErr: masked error: runtime error: invalid memory address or nil pointer dereference
catch.go:29: *withstack.withStack (top exception)
*assert.withAssertionFailure
Stacktrace (expand for inline code snippets):

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 620 to 622 in 62ad175

childProps := BuildChildPhysicalPropsScalar(o.mem, scalar, i)
childCost, childOptimized := o.optimizeExpr(scalar.Child(i), childProps)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 310 to 312 in 62ad175

case opt.ScalarExpr:
return o.optimizeScalarExpr(t)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 620 to 622 in 62ad175

childProps := BuildChildPhysicalPropsScalar(o.mem, scalar, i)
childCost, childOptimized := o.optimizeExpr(scalar.Child(i), childProps)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 310 to 312 in 62ad175

case opt.ScalarExpr:
return o.optimizeScalarExpr(t)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 620 to 622 in 62ad175

childProps := BuildChildPhysicalPropsScalar(o.mem, scalar, i)
childCost, childOptimized := o.optimizeExpr(scalar.Child(i), childProps)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 298 to 300 in 62ad175

case memo.RelExpr:
state := o.optimizeGroup(t, required)
return state.cost, state.fullyOptimized

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 522 to 524 in 62ad175

// Optimize the group member with respect to the required properties.
memberOptimized := o.optimizeGroupMember(state, member, required)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 577 to 579 in 62ad175

// Optimize the child with respect to those properties.
childCost, childOptimized := o.optimizeExpr(member.Child(i), childRequired)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 298 to 300 in 62ad175

case memo.RelExpr:
state := o.optimizeGroup(t, required)
return state.cost, state.fullyOptimized

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 522 to 524 in 62ad175

// Optimize the group member with respect to the required properties.
memberOptimized := o.optimizeGroupMember(state, member, required)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 577 to 579 in 62ad175

// Optimize the child with respect to those properties.
childCost, childOptimized := o.optimizeExpr(member.Child(i), childRequired)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 298 to 300 in 62ad175

case memo.RelExpr:
state := o.optimizeGroup(t, required)
return state.cost, state.fullyOptimized

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 522 to 524 in 62ad175

// Optimize the group member with respect to the required properties.
memberOptimized := o.optimizeGroupMember(state, member, required)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 577 to 579 in 62ad175

// Optimize the child with respect to those properties.
childCost, childOptimized := o.optimizeExpr(member.Child(i), childRequired)

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 298 to 300 in 62ad175

case memo.RelExpr:
state := o.optimizeGroup(t, required)
return state.cost, state.fullyOptimized

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 535 to 537 in 62ad175

// other expressions in this group.
if o.shouldExplore(required) && !o.explorer.exploreGroup(grp, required).fullyExplored {
fullyOptimized = false

cockroach/pkg/sql/opt/xform/explorer.go

Lines 184 to 186 in 62ad175

if memberExplored := e.exploreGroupMember(state, member, i, required); memberExplored {
// No more rules can ever match this expression, so skip it in

https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/bazel-out/k8-opt/bin/pkg/sql/opt/xform/explorer.og.go#L27-L29
https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/bazel-out/k8-opt/bin/pkg/sql/opt/xform/explorer.og.go#L829-L831

cockroach/pkg/sql/opt/xform/join_funcs.go

Lines 260 to 262 in 62ad175

) {
c.generateLookupJoinsImpl(
grp, joinType,

cockroach/pkg/sql/opt/xform/join_funcs.go

Lines 399 to 401 in 62ad175

// have a `ScanExpr` with properties fully populated.
input2 = scanExpr.Memo().MemoizeScan(scanPrivate)
tabMeta := c.e.mem.Metadata().TableMeta(scanPrivate2.Table)

https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/bazel-out/k8-opt/bin/pkg/sql/opt/memo/expr.og.go#L19989-L19991

cockroach/pkg/sql/opt/memo/logical_props_builder.go

Lines 185 to 187 in 62ad175

if !b.disableStats {
b.sb.buildScan(scan, rel)
}

cockroach/pkg/sql/opt/memo/statistics_builder.go

Lines 792 to 794 in 62ad175

}
s.Available = sb.availabilityFromInput(scan)

cockroach/pkg/sql/opt/memo/statistics_builder.go

Lines 282 to 284 in 62ad175

case *ScanExpr:
return sb.makeTableStatistics(t.Table).Available

cockroach/pkg/sql/opt/memo/statistics_builder.go

Lines 588 to 590 in 62ad175

func (sb *statisticsBuilder) makeTableStatistics(tabID opt.TableID) *props.Statistics {
stats, ok := sb.md.TableAnnotation(tabID, statsAnnID).(*props.Statistics)
if ok {

cockroach/pkg/sql/opt/table_meta.go

Lines 551 to 553 in 62ad175

func (md *Metadata) TableAnnotation(tabID TableID, annID TableAnnID) interface{} {
return md.tables[tabID.index()].anns[annID]
}

GOROOT/src/runtime/signal_unix.go#L838-L840
GOROOT/src/runtime/panic.go#L259-L261
GOROOT/src/runtime/panic.go#L883-L885

cockroach/pkg/sql/opt/xform/optimizer.go

Lines 247 to 249 in 62ad175

// locks.
if ok, e := errorutil.ShouldCatch(r); ok {
err = e

cockroach/pkg/util/errorutil/catch.go

Lines 28 to 30 in 62ad175

// get reported to Sentry.
err = errors.HandleAsAssertionFailure(err)
} 

pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeScalarExpr at line 621
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeExpr at line 311
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeScalarExpr at line 621
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeExpr at line 311
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeScalarExpr at line 621
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeExpr at line 299
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroup at line 523
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroupMember at line 578
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeExpr at line 299
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroup at line 523
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroupMember at line 578
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeExpr at line 299
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroup at line 523
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroupMember at line 578
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeExpr at line 299
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).optimizeGroup at line 536
pkg/sql/opt/xform/explorer.go in pkg/sql/opt/xform.(*explorer).exploreGroup at line 185
bazel-out/k8-opt/bin/pkg/sql/opt/xform/explorer.og.go in pkg/sql/opt/xform.(*explorer).exploreGroupMember at line 28
bazel-out/k8-opt/bin/pkg/sql/opt/xform/explorer.og.go in pkg/sql/opt/xform.(*explorer).exploreInnerJoin at line 830
pkg/sql/opt/xform/join_funcs.go in pkg/sql/opt/xform.(*CustomFuncs).GenerateLookupJoins at line 261
pkg/sql/opt/xform/join_funcs.go in pkg/sql/opt/xform.(*CustomFuncs).generateLookupJoinsImpl at line 400
bazel-out/k8-opt/bin/pkg/sql/opt/memo/expr.og.go in pkg/sql/opt/memo.(*Memo).MemoizeScan at line 19990
pkg/sql/opt/memo/logical_props_builder.go in pkg/sql/opt/memo.(*logicalPropsBuilder).buildScanProps at line 186
pkg/sql/opt/memo/statistics_builder.go in pkg/sql/opt/memo.(*statisticsBuilder).buildScan at line 793
pkg/sql/opt/memo/statistics_builder.go in pkg/sql/opt/memo.(*statisticsBuilder).availabilityFromInput at line 283
pkg/sql/opt/memo/statistics_builder.go in pkg/sql/opt/memo.(*statisticsBuilder).makeTableStatistics at line 589
pkg/sql/opt/table_meta.go in pkg/sql/opt.(*Metadata).TableAnnotation at line 552
GOROOT/src/runtime/signal_unix.go in runtime.sigpanic at line 839
GOROOT/src/runtime/panic.go in runtime.panicmem at line 260
GOROOT/src/runtime/panic.go in runtime.gopanic at line 884
pkg/sql/opt/xform/optimizer.go in pkg/sql/opt/xform.(*Optimizer).Optimize.func1 at line 248
pkg/util/errorutil/catch.go in pkg/util/errorutil.ShouldCatch at line 29

Tags

Tag Value
Command start-single-node
Environment v23.1.11
Go Version go1.19.10
Platform darwin amd64
Distribution CCL
Cockroach Release v23.1.11
Cockroach SHA 62ad175
# of CPUs 10
# of Goroutines 403

Jira issue: CRDB-38836
@cockroach-sentry cockroach-sentry added O-sentry Originated from an in-the-wild panic report. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels May 17, 2024
@yuzefovich
Copy link
Member

dup of #104009

@exalate-issue-sync exalate-issue-sync bot changed the title Sentry: catch.go:29: runtime error: invalid memory address or nil pointer dereference (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/u... Sentry: catch.go:29: runtime error: invalid memory address or nil pointer dereference (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/u... May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yuzefovich @cockroach-sentry