This is the mail archive of the binutils@sourceware.org mailing list for the binutils project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: RFC: Add initial support for .NET Core dlls to objdump


Hi Nick,

Thanks for all your comments! They have been really helpful.

Nick Clifton <nickc@redhat.com> writes:

> I also think that since the gentestdlls.c program is part of the binutils
> testsuite it ought to live in the binutils/testsuite directory and not the
> binutils/ directory.

I see that some other programs - bfdtest1 and bfdtest2 - also work
similarly. I have left them in binutils instead of moving them into
binutils/testsuite.

> Also you need a mechanism to build the gentestdlls
> executable, and to bail out from the test if the executable cannot be created.

The changes to Makefile.am take care of this already, no? I did run
autoreconf locally in both binutils-gdb and in binutils-gdb/binutils. I
did not include the changes in my patch because it causes a large diff,
and I also seem to have a different version of automake (1.16.1 vs 1.15.1).

As I understand it, if gentestdlls fails to build, it fails the entire
binutils build. Is that okay?

Do I have to handle other architectures in the test specially? Will
objdump be able to handle dumping i386 PE/Coff on aarch64 platforms, for
example?

> PS.  I am off on PTO for two weeks, so I will not be able to respond to emails
> until later this month...

Hope you are enjoying your time off!

Cheers,
Omair

--
PGP Key: B157A9F0 (http://pgp.mit.edu/)
Fingerprint = 9DB5 2F0B FD3E C239 E108  E7BD DF99 7AF8 B157 A9F0
>From 47f5117e29679216f35d9e104ddb2fdbdfeed940 Mon Sep 17 00:00:00 2001
From: Omair Majid <omajid@redhat.com>
Date: Tue, 25 Jun 2019 18:03:42 -0400
Subject: [PATCH] Handle some PE/COFF files generated by .NET

The System.Runtime.dll files that get shipped with .NET Core 2.1 on
different platforms demonstrate an issue:

    $ objdump -x dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll
    objdump: dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll: file format not recognized

After this fix:

    $ ~/local/binutils/bin/objdump -x dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll
    dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll:   file format pei-x86-64
    dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll
    architecture: i386:x86-64, flags 0x0000012f:
    HAS_RELOC, EXEC_P, HAS_LINENO, HAS_DEBUG, HAS_LOCALS, D_PAGED

    Characteristics 0x2022
        executable
        large address aware
        DLL

    Time/Date               Wed Jun  5 14:49:41 2019
    Magic                   020b    (PE32+)
    ...

These PE files are regular PE/COFF files but have a different value for
IMAGE_FILE_HEADER.Machine to indicate it contains a native image that
targets a non-Windows platform. This value is OS dependent and varies
across systems (linux vs netbsd vs macos).

ECMA-335 does not mention these new values; it still insists that the
only valid value is 0x14c.

This commit also introduces a tool to generate handcrafted dll files
that can be used in tests to verify objdump parsing.
---
 bfd/coffcode.h                              |   8 ++
 bfd/peXXigen.c                              |   7 +
 binutils/Makefile.am                        |   5 +-
 binutils/testsuite/binutils-all/objdump.exp |  29 ++++
 binutils/testsuite/gentestdlls.c            | 151 ++++++++++++++++++++
 include/coff/i386.h                         |  10 ++
 include/coff/pe.h                           |   9 ++
 include/coff/x86_64.h                       |  12 +-
 8 files changed, 228 insertions(+), 3 deletions(-)
 create mode 100644 binutils/testsuite/gentestdlls.c

diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index c67bfbb0e6..6ec6baa9b3 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -2104,6 +2104,10 @@ coff_set_arch_mach_hook (bfd *abfd, void * filehdr)
 #endif
 #ifdef I386MAGIC
     case I386MAGIC:
+    case I386_APPLE_MAGIC:
+    case I386_FREEBSD_MAGIC:
+    case I386_LINUX_MAGIC:
+    case I386_NETBSD_MAGIC:
     case I386PTXMAGIC:
     case I386AIXMAGIC:		/* Danbury PS/2 AIX C Compiler.  */
     case LYNXCOFFMAGIC:
@@ -2112,6 +2116,10 @@ coff_set_arch_mach_hook (bfd *abfd, void * filehdr)
 #endif
 #ifdef AMD64MAGIC
     case AMD64MAGIC:
+    case AMD64_APPLE_MAGIC:
+    case AMD64_FREEBSD_MAGIC:
+    case AMD64_LINUX_MAGIC:
+    case AMD64_NETBSD_MAGIC:
       arch = bfd_arch_i386;
       machine = bfd_mach_x86_64;
       break;
diff --git a/bfd/peXXigen.c b/bfd/peXXigen.c
index 77fb4933e7..ee6da6480d 100644
--- a/bfd/peXXigen.c
+++ b/bfd/peXXigen.c
@@ -35,6 +35,13 @@
    "Peering Inside the PE: A Tour of the Win32 Portable Executable
    File Format", MSJ 1994, Volume 9.
 
+   The PE/PEI format is also used by .NET. ECMA-335 describes this:
+
+   "Standard ECMA-335 Common Language Infrastructure (CLI)", 6th Edition, June 2012.
+
+   This is also available at
+   https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-335.pdf.
+
    The *sole* difference between the pe format and the pei format is that the
    latter has an MSDOS 2.0 .exe header on the front that prints the message
    "This app must be run under Windows." (or some such).
diff --git a/binutils/Makefile.am b/binutils/Makefile.am
index 128494ca9e..de93ffa040 100644
--- a/binutils/Makefile.am
+++ b/binutils/Makefile.am
@@ -17,7 +17,7 @@
 # <http://www.gnu.org/licenses/>.
 #
 
-AUTOMAKE_OPTIONS = dejagnu no-dist foreign
+AUTOMAKE_OPTIONS = dejagnu no-dist foreign subdir-objects
 ACLOCAL_AMFLAGS = -I .. -I ../config -I ../bfd
 
 SUBDIRS = doc po
@@ -93,8 +93,9 @@ EXTRA_SCRIPTS = embedspu
 ## Test programs.
 BFDTEST1_PROG = bfdtest1
 BFDTEST2_PROG = bfdtest2
+GENTESTDLLS_PROG = testsuite/gentestdlls
 
-TEST_PROGS = $(BFDTEST1_PROG) $(BFDTEST2_PROG)
+TEST_PROGS = $(BFDTEST1_PROG) $(BFDTEST2_PROG) $(GENTESTDLLS_PROG)
 
 ## We need a special rule to install the programs which are built with
 ## -new, and to rename cxxfilt to c++filt.
diff --git a/binutils/testsuite/binutils-all/objdump.exp b/binutils/testsuite/binutils-all/objdump.exp
index beaf44f954..48098f5c69 100644
--- a/binutils/testsuite/binutils-all/objdump.exp
+++ b/binutils/testsuite/binutils-all/objdump.exp
@@ -732,6 +732,35 @@ if {[is_elf_format]} then {
     remote_file host delete $testfile3
 }
 
+# Test objdump on .NET assemblies (PE files)
+
+proc test_objdump_dotnet_assemblies {} {
+    global OBJDUMP
+    global base_dir
+
+    set test "dotnet-assemblies"
+
+    set got [binutils_run "$base_dir/testsuite/gentestdlls" "tmpdir"]
+    set want "wrote dotnet-linux-x86-64.dll"
+    if ![regexp $want $got] then {
+	fail "$test"
+    }
+
+    set want "file format pei-i386"
+    set got [binutils_run $OBJDUMP "-x tmpdir/simple-i386.dll"]
+    if ![regexp $want $got] then {
+	fail "$test"
+    }
+
+    set want "file format pei-x86-64"
+    set got [binutils_run $OBJDUMP "-x tmpdir/dotnet-linux-x86-64.dll"]
+    if ![regexp $want $got] then {
+	fail "$test"
+    }
+}
+
+test_objdump_dotnet_assemblies
+
 # Options which are not tested: -a -D -R -T -x -l --stabs
 # I don't see any generic way to test any of these other than -a.
 # Tests could be written for specific targets, and that should be done
diff --git a/binutils/testsuite/gentestdlls.c b/binutils/testsuite/gentestdlls.c
new file mode 100644
index 0000000000..bb7d94cdc6
--- /dev/null
+++ b/binutils/testsuite/gentestdlls.c
@@ -0,0 +1,151 @@
+/* Copyright (C) 2019 Free Software Foundation, Inc.
+
+   This file is part of GNU Binutils.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software
+   Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA
+   02110-1301, USA.  */
+
+
+/* This file generates a number of DLL (PE/COFF binaries traditionally
+   used on Windows) that we can then utilize in various tests to
+   ensure objdump can parse these file correctly.
+
+   See:
+   https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-335.pdf  */
+
+#include <memory.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+static void
+write_dos_header_and_stub (FILE* file)
+{
+  /* See ECMA-335 II.25.2.1.
+     Instead of lfanew, lets just hardcode the offset of the next byte
+     after this header (0x80).  */
+  char buffer[128] =
+    {
+     0x4d, 0x5a, 0x90, 0x00, 0x03, 0x00, 0x00, 0x00,
+     0x04, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0x00, 0x00,
+     0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+     0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+     0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, /* Last 4 bytes are precomputed lfanew.  */
+     0x0e, 0x1f, 0xba, 0x0e, 0x00, 0xb4, 0x09, 0xcd,
+     0x21, 0xb8, 0x01, 0x4c, 0xcd, 0x21, 0x54, 0x68,
+     0x69, 0x73, 0x20, 0x70, 0x72, 0x6f, 0x67, 0x72,
+     0x61, 0x6d, 0x20, 0x63, 0x61, 0x6e, 0x6e, 0x6f,
+     0x74, 0x20, 0x62, 0x65, 0x20, 0x72, 0x75, 0x6e,
+     0x20, 0x69, 0x6e, 0x20, 0x44, 0x4f, 0x53, 0x20,
+     0x6d, 0x6f, 0x64, 0x65, 0x2e, 0x0d, 0x0d, 0x0a,
+     0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+    };
+
+  fwrite (buffer, 1, 128, file);
+}
+
+static void
+write_pe_signature (FILE* file)
+{
+  char buffer[4];
+  buffer[0] = 'P';
+  buffer[1] = 'E';
+  buffer[2] = 0;
+  buffer[3] = 0;
+  fwrite (buffer, 1, 4, file);
+}
+
+static void
+write_coff_header (FILE* file, uint16_t machine)
+{
+  char buffer[128];
+  memset (buffer, 0, sizeof (buffer));
+
+  /* Machine.  ECMA-335 says this must be 0x14c but that's not true anymore.  */
+  buffer[0] = machine & 0xff;
+  buffer[1] = machine >> 0x8;
+  fwrite (buffer, 2, 1, file);
+  memset (buffer, 0, sizeof (buffer));
+  /* NumberOfSections = 0 */
+  fwrite (buffer, 2, 1, file);
+  /* TimeDateStamp = 0 */
+  fwrite (buffer, 4, 1, file);
+  /* PointerToSymbolTable = 0 */
+  fwrite (buffer, 4, 1, file);
+  /* NumberOfSymbols = 0 */
+  fwrite (buffer, 4, 1, file);
+  /* OptionalHeaderSize = 0 */
+  fwrite (buffer, 2, 1, file);
+  /* Characteristics = 0x2000 */
+  buffer[0] = 0x00;
+  buffer[1] = 0x20;
+  fwrite (buffer, 2, 1, file);
+  memset (buffer, 0 , sizeof (buffer));
+}
+
+int
+main(int argc, char** argv)
+{
+  if (argc < 2)
+    {
+      fprintf (stderr, "usage: %s output-directory\n", argv[0]);
+      exit (2);
+    }
+  if (chdir (argv[1]) != 0)
+    {
+      fprintf (stderr, "error: unable to change directory to %s\n", argv[0]);
+      exit (2);
+    }
+
+  FILE* file;
+
+  /* Generate a simple DLL file.  */
+  file = fopen ("simple-i386.dll", "w");
+  if (file == NULL)
+    {
+      fprintf (stderr, "error: unable to open file for writing\n");
+      exit (1);
+    }
+
+  write_dos_header_and_stub (file);
+  write_pe_signature (file);
+  write_coff_header (file, 0x14c);
+  fclose (file);
+  printf ("wrote simple-i386.dll\n");
+
+  /* Generate a sample .NET Core on Linux dll file.  As opposed to the
+     more common DLLs that contain bytecode (CIL/MSIL), many .NET Core
+     DLLs are pre-compiled for specific architectures and platforms.
+     See https://github.com/jbevain/cecil/issues/337 for an example of
+     this value being used in practice.  */
+  file = fopen ("dotnet-linux-x86-64.dll", "w");
+  if (file == NULL)
+    {
+      fprintf (stderr, "error: unable to open file for writing\n");
+      exit (1);
+    }
+
+  write_dos_header_and_stub (file);
+  write_pe_signature (file);
+  write_coff_header (file, 0xfd1d /* x86-64 + Linux */);
+  fclose (file);
+  printf ("wrote dotnet-linux-x86-64.dll\n");
+
+  return 0;
+}
diff --git a/include/coff/i386.h b/include/coff/i386.h
index 1d2ccff6f7..9ea7ddfa8a 100644
--- a/include/coff/i386.h
+++ b/include/coff/i386.h
@@ -43,7 +43,17 @@
 
 #define LYNXCOFFMAGIC	0415
 
+/* Used in some .NET DLLs that target a specific OS.  */
+#define I386_APPLE_MAGIC (I386MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE)
+#define I386_FREEBSD_MAGIC (I386MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_FREEBSD_OVERRIDE)
+#define I386_LINUX_MAGIC (I386MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE)
+#define I386_NETBSD_MAGIC (I386MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_NETBSD_OVERRIDE)
+
 #define I386BADMAG(x) (  ((x).f_magic != I386MAGIC) \
+		       && (x).f_magic != I386_APPLE_MAGIC \
+		       && (x).f_magic != I386_FREEBSD_MAGIC \
+		       && (x).f_magic != I386_LINUX_MAGIC \
+		       && (x).f_magic != I386_NETBSD_MAGIC \
 		       && (x).f_magic != I386AIXMAGIC \
 		       && (x).f_magic != I386PTXMAGIC \
 		       && (x).f_magic != LYNXCOFFMAGIC)
diff --git a/include/coff/pe.h b/include/coff/pe.h
index 85cc331831..8b3251e587 100644
--- a/include/coff/pe.h
+++ b/include/coff/pe.h
@@ -158,6 +158,15 @@
 #define IMAGE_FILE_MACHINE_WCEMIPSV2         0x0169
 #define IMAGE_FILE_MACHINE_AMD64             0x8664
 
+/* .NET DLLs XOR the Machine number (above) with an override to
+    indicate that the DLL contains OS-specific machine code rather
+    than just IL or bytecode. See
+    https://github.com/dotnet/coreclr/blob/6f7aa7967c607b8c667518314ab937c0d7547025/src/inc/pedecoder.h#L94-L107. */
+#define IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE 0x4644
+#define IMAGE_FILE_MACHINE_NATIVE_FREEBSD_OVERRIDE 0xadc4
+#define IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE 0x7b79
+#define IMAGE_FILE_MACHINE_NATIVE_NETBSD_OVERRIDE 0x1993
+
 #define IMAGE_SUBSYSTEM_UNKNOWN			 0
 #define IMAGE_SUBSYSTEM_NATIVE			 1
 #define IMAGE_SUBSYSTEM_WINDOWS_GUI		 2
diff --git a/include/coff/x86_64.h b/include/coff/x86_64.h
index 3d0e6f085d..5dde1337d5 100644
--- a/include/coff/x86_64.h
+++ b/include/coff/x86_64.h
@@ -28,8 +28,18 @@
 #define COFF_PAGE_SIZE	0x1000
 
 #define AMD64MAGIC	0x8664
+/* Used in some .NET DLLs that target a specific OS.  */
+#define AMD64_APPLE_MAGIC (AMD64MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE)
+#define AMD64_FREEBSD_MAGIC (AMD64MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_FREEBSD_OVERRIDE)
+#define AMD64_LINUX_MAGIC (AMD64MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE)
+#define AMD64_NETBSD_MAGIC (AMD64MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_NETBSD_OVERRIDE)
+
+#define AMD64BADMAG(x) (   ((x).f_magic != AMD64MAGIC) \
+                        && ((x).f_magic != AMD64_APPLE_MAGIC) \
+                        && ((x).f_magic != AMD64_FREEBSD_MAGIC) \
+                        && ((x).f_magic != AMD64_LINUX_MAGIC) \
+                        && ((x).f_magic != AMD64_NETBSD_MAGIC))
 
-#define AMD64BADMAG(x)	((x).f_magic != AMD64MAGIC)
 #define IMAGE_NT_OPTIONAL_HDR64_MAGIC      0x20b
 
 #define OMAGIC          0404    /* Object files, eg as output.  */
-- 
2.21.0


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]