[PATCH] [BZ 21005] Add support for Intel 64 rdrand and rdseed record/replay

[Luis Machado]

The following patch addresses BZ 21005, which is gdb failing to
recognize an rdrand instruction.

It enables support for both rdrand and rdseed and handles extended register
addressing (R8~R15) for 16-bit, 32-bit and 64-bit.

I tested this by hand with quite a few lines of inline asm.  I thought we
had arch-specific tests in testsuite/, but it looks like we don't exercise
every little instruction with record/replay, so no testcase.  Let me know
if one is required.

Regression-tested on Ubuntu 16.04 x86-64.

OK?

gdb/ChangeLog

2017-01-13  Luis Machado  <lgustavo@codesourcery.com>

	* NEWS: Mention support for record/replay of Intel 64 rdrand and
	rdseed instructions.
	i386-tdep.c (i386_process_record): Handle Intel 64 rdrand and rseed.
---
 gdb/NEWS        |  3 +++
 gdb/i386-tdep.c | 29 +++++++++++++++++++++++++----
 2 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/gdb/NEWS b/gdb/NEWS
index b976815..2429148 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -3,6 +3,9 @@
 
 *** Changes since GDB 7.12
 
+* GDB now supports recording and replaying rdrand and rdseed Intel 64
+  instructions.
+
 * Building GDB and GDBserver now requires a C++11 compiler.
 
   For example, GCC 4.8 or later.
diff --git a/gdb/i386-tdep.c b/gdb/i386-tdep.c
index 8a4d59f..5c7c2bc 100644
--- a/gdb/i386-tdep.c
+++ b/gdb/i386-tdep.c
@@ -5501,14 +5501,35 @@ i386_process_record (struct gdbarch *gdbarch, struct regcache *regcache,
       I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_EFLAGS_REGNUM);
       break;
 
-    case 0x0fc7:    /* cmpxchg8b */
+    case 0x0fc7:    /* cmpxchg8b / rdrand / rdseed */
       if (i386_record_modrm (&ir))
 	return -1;
       if (ir.mod == 3)
 	{
-	  ir.addr -= 2;
-	  opcode = opcode << 8 | ir.modrm;
-	  goto no_support;
+	  /* rdrand and rdseed use the 3 bits of the REG field of ModR/M as
+	     an extended opcode.  rdrand has bits 110 (/6) and rdseed
+	     has bits 111 (/7).  */
+	  if (ir.reg == 6 || ir.reg == 7)
+	    {
+	      /* The storage register is described by the 3 R/M bits, but the
+		 REX.B prefix may be used to give access to registers
+		 R8~R15.
+
+		 REX.W may also be used to access 64-bit registers, but we
+		 already record entire registers and not just partial bits
+		 of them.  */
+	      I386_RECORD_FULL_ARCH_LIST_ADD_REG (ir.rex_b ? ir.rex_b : ir.rm);
+	      /* These instructions also set conditional bits.  */
+	      I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_EFLAGS_REGNUM);
+	      break;
+	    }
+	  else
+	    {
+	      /* We don't handle this particular instruction yet.  */
+	      ir.addr -= 2;
+	      opcode = opcode << 8 | ir.modrm;
+	      goto no_support;
+	    }
 	}
       I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_REAX_REGNUM);
       I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_REDX_REGNUM);
-- 
2.7.4