Skip to content

Commit b358319

Browse files
committed
Auto merge of #33612 - royalstream:royalstream-enc-enum-ptr, r=michaelwoerister
gdb Pretty Print: generic encoded was failing on reference/pointer types If you debug this program using **gdb** ```rust fn main() { let x = 10; let y = Some(&x); // additional code } ``` And you try to print **y**'s value from the debugger, you get the following: ``` (gdb) print y Python Exception <class 'gdb.error'> Cannot convert value to int.: $1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}} ``` What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails. Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration. After this modification **gdb** prints **y** like this: ``` (gdb) print y $1 = Some = {0x7fff5fbff97c} ``` Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.
2 parents 1a26d23 + 04baf25 commit b358319

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

src/etc/gdb_rust_pretty_printing.py

+2
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,8 @@ def get_child_at_index(self, index):
7070
return child
7171

7272
def as_integer(self):
73+
if self.gdb_val.type.code == gdb.TYPE_CODE_PTR:
74+
return int(str(self.gdb_val), 0)
7375
return int(self.gdb_val)
7476

7577
def get_wrapped_value(self):

0 commit comments

Comments
 (0)