In arm-tdep.c arm_mem_r is declared as the following:

/* ARM memory record structure.  */
struct arm_mem_r
{
  uint32_t len;    /* Record length.  */
  CORE_ADDR addr;   /* Memory address.  */
};

In various places the content is first initialized via the directly-addressed record_buf_mem buffer:

          uint32_t record_buf_mem[8];
          :
          record_buf_mem[0] = 4;
          record_buf_mem[1] = tgt_mem_addr;

and copied over via the MEM_ALLOC macro:

#define MEM_ALLOC(MEMS, LENGTH, RECORD_BUF) \
        do  \
          { \
            unsigned int mem_len = LENGTH; \
            if (mem_len) \
            { \
              MEMS =  XNEWVEC (struct arm_mem_r, mem_len);  \
              memcpy(&MEMS->len, &RECORD_BUF[0], \
                     sizeof(struct arm_mem_r) * LENGTH); \
            } \
          } \
          while (0)

The problem is that CORE_ADDR is declared as long and on 64-bit host it is 8 byte. Because of that record_buf_mem[1] is no longer the starting address of the addr field but a padding space. Here is the memory dump through top-gdb:

(top-gdb) p /x arm_record.arm_mems[0]
$23 = {len = 0x4, addr = 0xe5832000}
(top-gdb) x /4x arm_record.arm_mems
0xb7f8d0:	0x00000004	0x0109e020	0xe5832000	0x00000000

where 0x0109e020 is the desired arm_record.arm_mems[0].addr value.

My temporary hack in my project is to declare thje addr field as uint32_t, but I think it is not a generic fix and will cause problems for aarch64.