RFC: Add initial support for .NET Core dlls to objdump
Omair Majid
omajid@redhat.com
Thu Jul 4 04:43:00 GMT 2019
Hi Nick,
Thanks for reviewing the patch and the great feedback!
* Nick Clifton <nickc@redhat.com> [2019-07-01 08:11]:
> > 0. Should this "non-stanard" magic field in the dll be exposed somewhere
> > in the objdump UI?
>
> Yes. Probably in the output for -p/--private-headers would be best. Have
> a look at the _bfd_XX_print_private_bfd_data_common function in bfd/peXXigen.c.
I am running into a bit of a wall here. I need to access the original
value of internal_filehdr->f_magic here. But that's not accessible at
this point.
I took a look at dump_bfd_header in objdump.c and that works with
processed bdf entries too.
Any pointers on how I can get a hold of the original f_magic field?
> > 1. Should I add tests for these? If so, any pointers on how to do that?
>
> If it is easy to do then yes. But I suspect that it might be difficult
> to construct a file containing the new machine field value. If you can
> do that however then the best place to add one or more tests would be the
> binutils/testsuite/binutils-all directory. Possibly as an addition to the
> objdump.exp file, or else as a new stand alone test.
Done. At least two dll files are now generated "by hand" when tests are
run.
> > 2. I added the new flags for architecture/OS combination for the binaries I
> > could find. Should I try and find out what the magic flags for other
> > architecture/OS combinations (bsds? arm64?) are? Even if I don't have
> > access to binary dlls that demonstrate this?
>
> Yes please. In cases like this it is best to try to cover as many variants
> as possible, even if you cannot test them all. If there are any errors then
> at some point in the future someone will file a bug report pointing out the
> problem. But if you get the values right first time then everyone will be
> happy, and no one will have to be nailed to a tree for doing it. (Sorry -
> slipped into misquoting the Hitch Hikers' Guide to the Galaxy there).
If it's okay with you, can I do that as a separate patch? I would like
to take care of take care of aarch64 (and others platforms and
architectures) separately instead of making this patch much larger.
> One thing missing from the patch, which is probably not your fault at all,
> is comment containing a URL to the official documentation for these new
> values. I am of course assuming that there is some official documentation,
> and that it is publicly accessible...
I added some links to some sources I have been looking at, but even
ECMA-335 says that only one valid value is possible, which we know is
no longer true. The best and the most concise source I can find is bug
report for a project:
https://github.com/jbevain/cecil/issues/337
The .NET Core sources are available under the MIT license:
https://github.com/dotnet/coreclr/blob/6f7aa7967c607b8c667518314ab937c0d7547025/src/inc/pedecoder.h#L94-L107
Is that something I can use/link to as "a source of
truth/documentation"?
I have attached an updated patch.
bfd/ChangeLog:
2019-07-03 Omair Majid <omajid@redhat.com>
* coffcode.h (coff_set_arch_mach_hook): Handle
I386_NATIVE_LINUX_MAGIC, I386_NATIVE_APPLE_MAGIC,
AMD64_NATIVE_LINUX_MAGIC and AMD64_NATIVE_LINUX_MAGIC.
* peXXigen.c: Add references to ECMA-335.
binutils/ChangeLog:
2019-07-03 Omair Majid <omajid@redhat.com>
* binutils/Makefile.amh (GENTESTDLLS_PROG): Define.
(TEST_PROGS): Add GENTESTDLLS_PROG.
(gentestdlls_DEPENDENCIES): Define.
* gentestdlls.c: New file.
* testsuite/binutils-all/objdump.exp
(test_objdump_dotnet_assemblies): Define.
include/ChangeLog:
2019-07-03 Omair Majid <omajid@redhat.com>
* coff/pe.h (IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE),
(IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE): Define.
* coff/i386.h (I386_NATIVE_LINUX_MAGIC),
(I386_NATIVE_APPLE_MAGIC): Define.
(I386BADMAG): Extend to include the above.
* coff/x86_64.h (AMD64_NATIVE_LINUX_MAGIC),
(AMD64_NATIVE_APPLE_MAGIC): Define.
(AMD64BADMAG): Extend to include the above.
Omair
--
PGP Key: B157A9F0 (http://pgp.mit.edu/)
Fingerprint = 9DB5 2F0B FD3E C239 E108 E7BD DF99 7AF8 B157 A9F0
-------------- next part --------------
>From 391a2ebbc34e9100c1c7e307b4415cb04b8b2c59 Mon Sep 17 00:00:00 2001
From: Omair Majid <omajid@redhat.com>
Date: Tue, 25 Jun 2019 18:03:42 -0400
Subject: [PATCH] Handle some PE/COFF files generated by .NET
The System.Runtime.dll files that get shipped with .NET Core 2.1 on
different platforms demonstrate an issue:
$ objdump -x dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll
objdump: dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll: file format not recognized
After this fix:
$ ~/local/binutils/bin/objdump -x dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll
dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll: file format pei-x86-64
dotnet/shared/Microsoft.NETCore.App/2.1.11/System.Runtime.dll
architecture: i386:x86-64, flags 0x0000012f:
HAS_RELOC, EXEC_P, HAS_LINENO, HAS_DEBUG, HAS_LOCALS, D_PAGED
Characteristics 0x2022
executable
large address aware
DLL
Time/Date Wed Jun 5 14:49:41 2019
Magic 020b (PE32+)
...
These PE files are regular PE/COFF files but have a different value for
IMAGE_FILE_HEADER.Machine to indicate it contains a native image that
targets a non-Windows platform. This value is OS dependent and varies
across systems (linux vs netbsd vs macos).
ECMA-335 does not mention these new values; it still insists that the
only valid value is 0x14c.
This commit also introduces a tool to generate handcrafted dll files
that can be used in tests to verify objdump parsing.
---
bfd/coffcode.h | 4 +
bfd/peXXigen.c | 7 +
binutils/Makefile.am | 4 +-
binutils/gentestdlls.c | 167 ++++++++++++++++++++
binutils/testsuite/binutils-all/objdump.exp | 29 ++++
include/coff/i386.h | 6 +
include/coff/pe.h | 3 +
include/coff/x86_64.h | 8 +-
8 files changed, 226 insertions(+), 2 deletions(-)
create mode 100644 binutils/gentestdlls.c
diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index c67bfbb0e6..94f0bacc4a 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -2104,6 +2104,8 @@ coff_set_arch_mach_hook (bfd *abfd, void * filehdr)
#endif
#ifdef I386MAGIC
case I386MAGIC:
+ case I386_NATIVE_LINUX_MAGIC:
+ case I386_NATIVE_APPLE_MAGIC:
case I386PTXMAGIC:
case I386AIXMAGIC: /* Danbury PS/2 AIX C Compiler. */
case LYNXCOFFMAGIC:
@@ -2112,6 +2114,8 @@ coff_set_arch_mach_hook (bfd *abfd, void * filehdr)
#endif
#ifdef AMD64MAGIC
case AMD64MAGIC:
+ case AMD64_NATIVE_LINUX_MAGIC:
+ case AMD64_NATIVE_APPLE_MAGIC:
arch = bfd_arch_i386;
machine = bfd_mach_x86_64;
break;
diff --git a/bfd/peXXigen.c b/bfd/peXXigen.c
index 77fb4933e7..ee6da6480d 100644
--- a/bfd/peXXigen.c
+++ b/bfd/peXXigen.c
@@ -35,6 +35,13 @@
"Peering Inside the PE: A Tour of the Win32 Portable Executable
File Format", MSJ 1994, Volume 9.
+ The PE/PEI format is also used by .NET. ECMA-335 describes this:
+
+ "Standard ECMA-335 Common Language Infrastructure (CLI)", 6th Edition, June 2012.
+
+ This is also available at
+ https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-335.pdf.
+
The *sole* difference between the pe format and the pei format is that the
latter has an MSDOS 2.0 .exe header on the front that prints the message
"This app must be run under Windows." (or some such).
diff --git a/binutils/Makefile.am b/binutils/Makefile.am
index 128494ca9e..625a3fd7e6 100644
--- a/binutils/Makefile.am
+++ b/binutils/Makefile.am
@@ -93,8 +93,9 @@ EXTRA_SCRIPTS = embedspu
## Test programs.
BFDTEST1_PROG = bfdtest1
BFDTEST2_PROG = bfdtest2
+GENTESTDLLS_PROG = gentestdlls
-TEST_PROGS = $(BFDTEST1_PROG) $(BFDTEST2_PROG)
+TEST_PROGS = $(BFDTEST1_PROG) $(BFDTEST2_PROG) $(GENTESTDLLS_PROG)
## We need a special rule to install the programs which are built with
## -new, and to rename cxxfilt to c++filt.
@@ -233,6 +234,7 @@ elfedit_DEPENDENCIES = $(LIBINTL_DEP) $(LIBIBERTY)
dllwrap_DEPENDENCIES = $(LIBINTL_DEP) $(LIBIBERTY)
bfdtest1_DEPENDENCIES = $(LIBINTL_DEP) $(LIBIBERTY) $(BFDLIB)
bfdtest2_DEPENDENCIES = $(LIBINTL_DEP) $(LIBIBERTY) $(BFDLIB)
+gentestdlls_DEPENDENCIES =
LDADD = $(BFDLIB) $(LIBIBERTY) $(LIBINTL)
diff --git a/binutils/gentestdlls.c b/binutils/gentestdlls.c
new file mode 100644
index 0000000000..ff2d6add7e
--- /dev/null
+++ b/binutils/gentestdlls.c
@@ -0,0 +1,167 @@
+/* Copyright (C) 2007-2019 Free Software Foundation, Inc.
+
+ This file is part of GNU Binutils.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA
+ 02110-1301, USA. */
+
+
+/*
+ This file generates a number of DLL (PE/COFF binaries traditionally
+ used on Windows) that we can then utilize in various tests to
+ ensure objdump can parse these file correctly.
+
+ See:
+ https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-335.pdf
+ https://docs.microsoft.com/en-us/windows/desktop/debug/pe-format
+ https://en.wikibooks.org/wiki/X86_Disassembly/Windows_Executable_Files
+
+ */
+
+#include <memory.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+static int write_dos_header_and_stub(FILE* file);
+static int write_pe_signature(FILE* file);
+static int write_coff_header(FILE* file, uint16_t machine);
+
+static int
+write_dos_header_and_stub(FILE* file)
+{
+ /*
+ * See ECMA-335 II.25.2.1.
+ *
+ * Instead of lfanew, lets just hardcode the offset of the next byte
+ * after this header (0x80).
+ */
+ char buffer[128] = {
+ 0x4d, 0x5a, 0x90, 0x00, 0x03, 0x00, 0x00, 0x00,
+ 0x04, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0x00, 0x00,
+ 0xb8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, /* Last 4 bytes are precomputed lfanew. */
+ 0x0e, 0x1f, 0xba, 0x0e, 0x00, 0xb4, 0x09, 0xcd,
+ 0x21, 0xb8, 0x01, 0x4c, 0xcd, 0x21, 0x54, 0x68,
+ 0x69, 0x73, 0x20, 0x70, 0x72, 0x6f, 0x67, 0x72,
+ 0x61, 0x6d, 0x20, 0x63, 0x61, 0x6e, 0x6e, 0x6f,
+ 0x74, 0x20, 0x62, 0x65, 0x20, 0x72, 0x75, 0x6e,
+ 0x20, 0x69, 0x6e, 0x20, 0x44, 0x4f, 0x53, 0x20,
+ 0x6d, 0x6f, 0x64, 0x65, 0x2e, 0x0d, 0x0d, 0x0a,
+ 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
+ };
+
+ fwrite(buffer, 1, 128, file);
+
+ return 0;
+}
+
+static int
+write_pe_signature(FILE* file)
+{
+ char buffer[4];
+ buffer[0] = 'P';
+ buffer[1] = 'E';
+ buffer[2] = 0;
+ buffer[3] = 0;
+ fwrite(buffer, 1, 4, file);
+ return 0;
+}
+
+static int
+write_coff_header(FILE* file, uint16_t machine)
+{
+ char buffer[128];
+ memset(buffer, 0, sizeof(buffer));
+
+ /* Machine. ECMA-335 says this must be 0x14c but that's not true anymore. */
+ buffer[0] = machine & 0xff;
+ buffer[1] = machine >> 0x8;
+ fwrite(buffer, 2, 1, file);
+ memset(buffer, 0, sizeof(buffer));
+ /* NumberOfSections = 0 */
+ fwrite(buffer, 2, 1, file);
+ /* TimeDateStamp = 0 */
+ fwrite(buffer, 4, 1, file);
+ /* PointerToSymbolTable = 0 */
+ fwrite(buffer, 4, 1, file);
+ /* NumberOfSymbols = 0 */
+ fwrite(buffer, 4, 1, file);
+ /* OptionalHeaderSize = 0 */
+ fwrite(buffer, 2, 1, file);
+ /* Characteristics = 0x2000 */
+ buffer[0] = 0x00;
+ buffer[1] = 0x20;
+ fwrite(buffer, 2, 1, file);
+ memset(buffer, 0 , sizeof(buffer));
+
+ return 0;
+}
+
+int
+main(int argc, char** argv)
+{
+ if (argc < 2)
+ {
+ fprintf(stderr, "usage: %s output-directory\n", argv[0]);
+ exit(2);
+ }
+ if (chdir(argv[1]) != 0)
+ {
+ fprintf(stderr, "error: unable to change directory to %s\n", argv[0]);
+ exit(2);
+ }
+
+ FILE* file;
+
+ /* Generate a simple DLL file. */
+ file = fopen("simple-i386.dll", "w");
+ if (file == NULL)
+ {
+ fprintf(stderr, "error: unable to open file for writing\n");
+ exit(1);
+ }
+
+ write_dos_header_and_stub(file);
+ write_pe_signature(file);
+ write_coff_header(file, 0x14c);
+ fclose(file);
+ printf("wrote simple-i386.dll\n");
+
+ /* Generate a sample .NET Core on Linux dll file. As opposed to the
+ more common DLLs that contain bytecode (CIL/MSIL), many .NET Core
+ DLLs are pre-compiled for specific architectures and platforms.
+ See https://github.com/jbevain/cecil/issues/337 for an example of
+ this value being used in practice. */
+ file = fopen("dotnet-linux-x86-64.dll", "w");
+ if (file == NULL)
+ {
+ fprintf(stderr, "error: unable to open file for writing\n");
+ exit(1);
+ }
+
+ write_dos_header_and_stub(file);
+ write_pe_signature(file);
+ write_coff_header(file, 0xfd1d /* x86-64 + Linux */);
+ fclose(file);
+ printf("wrote dotnet-linux-x86-64.dll\n");
+
+ return 0;
+}
diff --git a/binutils/testsuite/binutils-all/objdump.exp b/binutils/testsuite/binutils-all/objdump.exp
index beaf44f954..657b4541c8 100644
--- a/binutils/testsuite/binutils-all/objdump.exp
+++ b/binutils/testsuite/binutils-all/objdump.exp
@@ -732,6 +732,35 @@ if {[is_elf_format]} then {
remote_file host delete $testfile3
}
+# Test objdump on .NET assemblies (PE files)
+
+proc test_objdump_dotnet_assemblies {} {
+ global OBJDUMP
+ global base_dir
+
+ set test "dotnet-assemblies"
+
+ set got [binutils_run "$base_dir/gentestdlls" "tmpdir"]
+ set want "wrote dotnet-linux-x86-64.dll"
+ if ![regexp $want $got] then {
+ fail "$test"
+ }
+
+ set want "file format pei-i386"
+ set got [binutils_run $OBJDUMP "-x tmpdir/simple-i386.dll"]
+ if ![regexp $want $got] then {
+ fail "$test"
+ }
+
+ set want "file format pei-x86-64"
+ set got [binutils_run $OBJDUMP "-x tmpdir/dotnet-linux-x86-64.dll"]
+ if ![regexp $want $got] then {
+ fail "$test"
+ }
+}
+
+test_objdump_dotnet_assemblies
+
# Options which are not tested: -a -D -R -T -x -l --stabs
# I don't see any generic way to test any of these other than -a.
# Tests could be written for specific targets, and that should be done
diff --git a/include/coff/i386.h b/include/coff/i386.h
index 1d2ccff6f7..55b67572bf 100644
--- a/include/coff/i386.h
+++ b/include/coff/i386.h
@@ -43,7 +43,13 @@
#define LYNXCOFFMAGIC 0415
+/* Used in .NET DLLs that target a specific platform. */
+#define I386_NATIVE_LINUX_MAGIC (I386MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE)
+#define I386_NATIVE_APPLE_MAGIC (I386MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE)
+
#define I386BADMAG(x) ( ((x).f_magic != I386MAGIC) \
+ && (x).f_magic != I386_NATIVE_LINUX_MAGIC \
+ && (x).f_magic != I386_NATIVE_APPLE_MAGIC \
&& (x).f_magic != I386AIXMAGIC \
&& (x).f_magic != I386PTXMAGIC \
&& (x).f_magic != LYNXCOFFMAGIC)
diff --git a/include/coff/pe.h b/include/coff/pe.h
index 85cc331831..c401586b2e 100644
--- a/include/coff/pe.h
+++ b/include/coff/pe.h
@@ -158,6 +158,9 @@
#define IMAGE_FILE_MACHINE_WCEMIPSV2 0x0169
#define IMAGE_FILE_MACHINE_AMD64 0x8664
+#define IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE 0x7b79
+#define IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE 0x4644
+
#define IMAGE_SUBSYSTEM_UNKNOWN 0
#define IMAGE_SUBSYSTEM_NATIVE 1
#define IMAGE_SUBSYSTEM_WINDOWS_GUI 2
diff --git a/include/coff/x86_64.h b/include/coff/x86_64.h
index 3d0e6f085d..d91845bd1b 100644
--- a/include/coff/x86_64.h
+++ b/include/coff/x86_64.h
@@ -28,8 +28,14 @@
#define COFF_PAGE_SIZE 0x1000
#define AMD64MAGIC 0x8664
+/* Used in .NET DLLs that target a specific platform. */
+#define AMD64_NATIVE_LINUX_MAGIC (AMD64MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE)
+#define AMD64_NATIVE_APPLE_MAGIC (AMD64MAGIC ^ IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE)
+
+#define AMD64BADMAG(x) (((x).f_magic != AMD64MAGIC) \
+ && ((x).f_magic != AMD64_NATIVE_LINUX_MAGIC) \
+ && ((x).f_magic != AMD64_NATIVE_APPLE_MAGIC))
-#define AMD64BADMAG(x) ((x).f_magic != AMD64MAGIC)
#define IMAGE_NT_OPTIONAL_HDR64_MAGIC 0x20b
#define OMAGIC 0404 /* Object files, eg as output. */
--
2.21.0
More information about the Binutils
mailing list