diff --git a/docs/design/conf.py b/docs/design/conf.py index 7c05e8082..f5814c4f4 100644 --- a/docs/design/conf.py +++ b/docs/design/conf.py @@ -33,6 +33,7 @@ 'sphinx.ext.coverage', 'sphinx.ext.mathjax', 'sphinx.ext.ifconfig', + 'myst_parser', # For now, do not use the githubpages extension # because the machines that run our buildbots do # not have this package installed @@ -45,8 +46,9 @@ # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # -# source_suffix = ['.rst', '.md'] -source_suffix = '.rst' +# source_suffix = ['.md'] +source_suffix = '.md' +myst_enable_extensions = ['colon_fence', 'deflist'] # The master toctree document. master_doc = 'index' @@ -71,7 +73,7 @@ # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. -language = None +language = 'en' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. diff --git a/docs/design/documentation/MemoryAreaRegionFragment.rst b/docs/design/documentation/MemoryAreaRegionFragment.md similarity index 56% rename from docs/design/documentation/MemoryAreaRegionFragment.rst rename to docs/design/documentation/MemoryAreaRegionFragment.md index f67286950..b9ccb5195 100644 --- a/docs/design/documentation/MemoryAreaRegionFragment.rst +++ b/docs/design/documentation/MemoryAreaRegionFragment.md @@ -1,11 +1,11 @@ -Memory Area Region Fragment -============================ +# Memory Area Region Fragment -.. contents:: - :local: +```{contents} +:local: true +``` + +## Information -Information ------------------ All input files that are placed on command line (scripts, archives and relocatables), have actual code/data and ELF specific structures. The actual code/data is stored as a RegionFragment in ELD. This fragment has a member @@ -20,42 +20,42 @@ waste of memory to keep the raw data throughtout the linking process. A more efficient representation of the RegionFragment will help reduce linking large code bases. -Option --------- +## Option + We will have an option called -Om that will reduce the memory usage due to input read only raw data. By default, this option is disabled. -Replacing raw data ------------------------ +## Replacing raw data By default the RegionFragment has an llvm::StringRef as member variable carrying the raw data. We can replace this with MemoryArea and abstract out if the raw data is kept on -file or in memory. The interface MemoryArea::request will use llvm::MemoryBuffer::getFile +file or in memory. The interface MemoryArea::request will use llvm::MemoryBuffer::getFile to open file seek a position for access by the caller. If the memory is in-memory, the operation should be a simple return of a StringRef. -API/class changes: -------------------- - RegionFragment: - Region fragment will have a pointer to a MemoryArea instead of a StringRef. - - RegionFragment::getRegion: - This will either return a StringRef or do a request and return a StringRef - from newly created memory buffer based on -Om option. - - MemoryArea: - The class will contain filename, offset in file where the region begins, - the size of region and a flag indicating if this area represents a file - or it is an "in memory" entity such as linker scripts created on the fly - with defsym options. - - MemoryArea::request: - API will open file and seek at offset in file + offset within region - supplied as parameter, create a StringRef of the same and return it. +## API/class changes: + +> RegionFragment: +> Region fragment will have a pointer to a MemoryArea instead of a StringRef. +> +> RegionFragment::getRegion: +> This will either return a StringRef or do a request and return a StringRef +> from newly created memory buffer based on -Om option. +> +> MemoryArea: +> The class will contain filename, offset in file where the region begins, +> the size of region and a flag indicating if this area represents a file +> or it is an "in memory" entity such as linker scripts created on the fly +> with defsym options. +> +> MemoryArea::request: +> API will open file and seek at offset in file + offset within region +> supplied as parameter, create a StringRef of the same and return it. + +## Future Enhancement(s) + +> We could use a user provided list of files to keep cached in memory. -Future Enhancement(s) ---------------------- - We could use a user provided list of files to keep cached in memory. diff --git a/docs/design/documentation/mergingSimilarStrings.rst b/docs/design/documentation/mergingSimilarStrings.md similarity index 75% rename from docs/design/documentation/mergingSimilarStrings.rst rename to docs/design/documentation/mergingSimilarStrings.md index aebb779e6..ff18af056 100644 --- a/docs/design/documentation/mergingSimilarStrings.rst +++ b/docs/design/documentation/mergingSimilarStrings.md @@ -1,13 +1,13 @@ -Merging Similar Strings -=========================== +# Merging Similar Strings -.. contents:: - :local: +```{contents} +:local: true +``` + +## Information -Information -------------- When the user uses strings that appear either in output/passed as parameters, they all go in to -readonly strings. Since it is a string, they are terminated by \0. +readonly strings. Since it is a string, they are terminated by 0. When the user uses constant data values that appear in function calls, they all go into readonly merge sections. Since its a data value, they all are associated @@ -17,8 +17,8 @@ To reduce the output memory foot print, they all need to be de-duped. This mode would be disabled during Partial Linking. -Option --------- +## Option + We will have an option called -Os thats associated with the linker, the merge strings option would be automatically turned on when the user uses this option. @@ -27,8 +27,7 @@ merge option will be disabled as part of that as well. By default, the merge option is disabled. -Handling AM Sections ------------------------ +## Handling AM Sections When the Linker needs to handle the AM section, they need to be split into multiple fragments defined by the Entry Size of the section. @@ -36,31 +35,30 @@ multiple fragments defined by the Entry Size of the section. Each fragment is associated with a symbol, and that is merged similar to how debug strings are merged. -The tricky part is to fix the relocations. +The tricky part is to fix the relocations. -Handling AMS Sections ------------------------ +## Handling AMS Sections When the Linker needs to handle the AMS section, they need to be split into -multiple fragments by looking at individual strings, that are terminated by \0. +multiple fragments by looking at individual strings, that are terminated by 0. Each fragment is associated with a symbol, and that is merged similar to how debug strings are merged. -Relocations -------------- +## Relocations + The relocations that are created would need to know how the individual fragments are split. To fix the relocation so that its associated with a proper fragment, the relocation offset and the addend information has to be modified so that they -point to +point to -a) The fragments when they have been split -b) Suffix/Prefix of the data thats used when the relocation is applied to the 'AM' section +1. The fragments when they have been split +2. Suffix/Prefix of the data thats used when the relocation is applied to the 'AM' section +## Diagnostics -Diagnostics ---------------- We need to print diagnostics on how many bytes were merged, and how much was -saved. +saved. + diff --git a/docs/design/index.md b/docs/design/index.md new file mode 100644 index 000000000..8a61a02a4 --- /dev/null +++ b/docs/design/index.md @@ -0,0 +1,20 @@ +% eld documentation + +# ELD Design Guide + +This document describes design details of ELD + +```{eval-rst} +.. toctree:: + :maxdepth: 2 + + documentation/mergingSimilarStrings.md + documentation/MemoryAreaRegionFragment.md +``` + +## Indices and tables + +- {ref}`genindex` +- {ref}`modindex` +- {ref}`search` + diff --git a/docs/design/index.rst b/docs/design/index.rst deleted file mode 100644 index 8c9237de8..000000000 --- a/docs/design/index.rst +++ /dev/null @@ -1,21 +0,0 @@ -.. eld documentation - -#################################################### -ELD Design Guide -#################################################### - -This document describes design details of ELD - -.. toctree:: - :maxdepth: 2 - - documentation/mergingSimilarStrings.rst - documentation/MemoryAreaRegionFragment.rst - -******************* -Indices and tables -******************* - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` diff --git a/docs/userguide/CMakeLists.txt b/docs/userguide/CMakeLists.txt index ef362815b..faae1dcae 100644 --- a/docs/userguide/CMakeLists.txt +++ b/docs/userguide/CMakeLists.txt @@ -42,7 +42,7 @@ if(LLVM_ENABLE_SPHINX) ${LINKER_OPTS_DUMP_DIR}/${target}LinkerOptionsTblGenDump.json DEPENDS eld-linkeroptions-create-json-dump-dir) - # Generates ${target}LinkerOptions restructureText documentation from the + # Generates ${target}LinkerOptions Markdown documentation from the # ${target}LinkerOptions JSON dump file. set(SKIP_OPTION "") if(NOT target MATCHES "Gnu") @@ -54,7 +54,7 @@ if(LLVM_ENABLE_SPHINX) COMMAND python3 ${CMAKE_CURRENT_SOURCE_DIR}/GenerateOptionsDocsFromTblGen.py ${LINKER_OPTS_DUMP_DIR}/${target}LinkerOptionsTblGenDump.json -o - ${DOCS_BUILD_SOURCE}/documentation/options/${target}LinkerOptions.rst + ${DOCS_BUILD_SOURCE}/documentation/options/${target}LinkerOptions.md ${SKIP_OPTION} -S ${OPTIONS_SUPPLEMENTS_DIR}) @@ -125,6 +125,7 @@ if(LLVM_ENABLE_SPHINX) add_custom_command( OUTPUT ${ELD_USERGUIDE_PDF} ${ELD_USERGUIDE_HTML_PDF} COMMAND + ${CMAKE_COMMAND} -E env --unset=PYTHONPATH ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/GenerateTypstPDF.py ${ELD_USERGUIDE_SINGLEHTML_DIR}/index.html ${ELD_USERGUIDE_TYPST} diff --git a/docs/userguide/CommandLineOptionsSupplements/remap-inputs-file.md.in b/docs/userguide/CommandLineOptionsSupplements/remap-inputs-file.md.in new file mode 100644 index 000000000..8a4ebe07d --- /dev/null +++ b/docs/userguide/CommandLineOptionsSupplements/remap-inputs-file.md.in @@ -0,0 +1,12 @@ +Read remap rules from ``. + +Each non-empty line in the file is one rule in either format: + +- `pattern=replacement` +- `pattern replacement` + +Lines beginning with `#` and blank lines are ignored. Inline `#` comments +are also supported. + +Rules from this file are appended to the same ordered remap list used by +{option}`--remap-inputs`. diff --git a/docs/userguide/CommandLineOptionsSupplements/remap-inputs-file.rst.in b/docs/userguide/CommandLineOptionsSupplements/remap-inputs-file.rst.in deleted file mode 100644 index 27a733627..000000000 --- a/docs/userguide/CommandLineOptionsSupplements/remap-inputs-file.rst.in +++ /dev/null @@ -1,12 +0,0 @@ -Read remap rules from ````. - -Each non-empty line in the file is one rule in either format: - -| ``pattern=replacement`` -| ``pattern replacement`` - -Lines beginning with ``#`` and blank lines are ignored. Inline ``#`` comments -are also supported. - -Rules from this file are appended to the same ordered remap list used by -:option:`--remap-inputs`. diff --git a/docs/userguide/CommandLineOptionsSupplements/remap-inputs.md.in b/docs/userguide/CommandLineOptionsSupplements/remap-inputs.md.in new file mode 100644 index 000000000..4d07a0aaa --- /dev/null +++ b/docs/userguide/CommandLineOptionsSupplements/remap-inputs.md.in @@ -0,0 +1,16 @@ +Rules are evaluated in command-line order; the first matching rule wins. +The pattern supports wildcard matching. + +Example: + +- `--remap-inputs=foo.o=bar.o` +- `--remap-inputs=*foo.o=build/alt/bar.o` + +Remapping is applied before path search/open, including linker-script +`INPUT(...)`, `GROUP(...)`, and `INCLUDE` processing. + +How to tell remapping is active: + +- `--verbose` prints: `Remapping input file to `. +- Text map output (`-MapStyle txt`) annotates remapped entries with + `# remapped from `. diff --git a/docs/userguide/CommandLineOptionsSupplements/remap-inputs.rst.in b/docs/userguide/CommandLineOptionsSupplements/remap-inputs.rst.in deleted file mode 100644 index 2f7d83436..000000000 --- a/docs/userguide/CommandLineOptionsSupplements/remap-inputs.rst.in +++ /dev/null @@ -1,16 +0,0 @@ -Rules are evaluated in command-line order; the first matching rule wins. -The pattern supports wildcard matching. - -Example: - -| ``--remap-inputs=foo.o=bar.o`` -| ``--remap-inputs=*foo.o=build/alt/bar.o`` - -Remapping is applied before path search/open, including linker-script -``INPUT(...)``, ``GROUP(...)``, and ``INCLUDE`` processing. - -How to tell remapping is active: - -| ``--verbose`` prints: ``Remapping input file to ``. -| Text map output (``-MapStyle txt``) annotates remapped entries with -| ``# remapped from ``. diff --git a/docs/userguide/CommandLineOptionsSupplements/sysroot.md.in b/docs/userguide/CommandLineOptionsSupplements/sysroot.md.in new file mode 100644 index 000000000..b285f8c26 --- /dev/null +++ b/docs/userguide/CommandLineOptionsSupplements/sysroot.md.in @@ -0,0 +1,20 @@ +- `INPUT(file, file, ...)` +- `GROUP(file, file, ...)` + +The files specified using the INPUT and GROUP command are searched inside the sysroot when: + +- The file begins with the `/` character, and +- The script containing the INPUT/GROUP command is located within the sysroot directory. + +Additionally, paths in INPUT/GROUP commands or on the command line can explicitly +force sysroot expansion by using the `=` or `$SYSROOT` prefix: + +- `=/path/to/file` expands to `/path/to/file` +- `$SYSROOT/path/to/file` expands to `/path/to/file` + +This works for both linker script commands and command-line input files. +If no sysroot is configured, the prefix is stripped and the remaining path is used. + +{option}`-L` `` + +When `` is prefixed with `=`, then `=` is expanded to the sysroot. diff --git a/docs/userguide/CommandLineOptionsSupplements/sysroot.rst.in b/docs/userguide/CommandLineOptionsSupplements/sysroot.rst.in deleted file mode 100644 index 1546be842..000000000 --- a/docs/userguide/CommandLineOptionsSupplements/sysroot.rst.in +++ /dev/null @@ -1,20 +0,0 @@ -| ``INPUT(file, file, ...)`` -| ``GROUP(file, file, ...)`` - -The files specified using the INPUT and GROUP command are searched inside the sysroot when: - - - The file begins with the ``/`` character, and - - The script containing the INPUT/GROUP command is located within the sysroot directory. - -Additionally, paths in INPUT/GROUP commands or on the command line can explicitly -force sysroot expansion by using the ``=`` or ``$SYSROOT`` prefix: - - - ``=/path/to/file`` expands to ``/path/to/file`` - - ``$SYSROOT/path/to/file`` expands to ``/path/to/file`` - -This works for both linker script commands and command-line input files. -If no sysroot is configured, the prefix is stripped and the remaining path is used. - -:option:`-L` ```` - -When ```` is prefixed with ``=``, then ``=`` is expanded to the sysroot. \ No newline at end of file diff --git a/docs/userguide/GenerateOptionsDocsFromTblGen.py b/docs/userguide/GenerateOptionsDocsFromTblGen.py index a68830716..dac8725cd 100755 --- a/docs/userguide/GenerateOptionsDocsFromTblGen.py +++ b/docs/userguide/GenerateOptionsDocsFromTblGen.py @@ -78,7 +78,7 @@ def generate_docs(self, out, supplement=None): option_forms = self.get_all_option_forms() if not option_forms: return - out.write(".. option:: " + ", ".join(option_forms)) + out.write("```{option} " + ", ".join(option_forms)) out.write("\n") if self.help_text: out.write("\n") @@ -86,19 +86,15 @@ def generate_docs(self, out, supplement=None): out.write("\n") if supplement: out.write("\n") - # Indent every line of the supplement by 3 spaces so it sits - # inside the .. option:: directive body. - indented = "\n".join( - " " + line if line.strip() else line - for line in supplement.splitlines() - ) - out.write(indented) - out.write("\n") + out.write(supplement) + if not supplement.endswith("\n"): + out.write("\n") + out.write("```\n") @classmethod def fixHelpTextFormatting(self, help_text): """ - Modifies help text formatting to be as per restructureText expectations. + Modifies help text formatting to be as per Markdown expectations. One of the complication transformations it performs is transforming documentation such as: @@ -119,7 +115,6 @@ def fixHelpTextFormatting(self, help_text): help_text = help_text.replace("\n", "\n\n", 1) help_text = re.sub(r"^[ \t]+-", r"* \g<0>", help_text, 0, re.MULTILINE) help_text = help_text.replace("\t", "") - help_text = " " * 3 + help_text.replace("\n", "\n" + " " * 3) return help_text @@ -143,9 +138,7 @@ def is_option_group(cls, raw_info) -> bool: ) def generate_docs(self, out): - group_line = self.help_text + "\n" - out.write(group_line) - out.write("^" * len(group_line) + "\n") + out.write("### " + self.help_text + "\n\n") def create_groups(raw_options_info): @@ -230,9 +223,9 @@ def create_argparser(): parser = argparse.ArgumentParser( prog="GenerateOptionsDocsFromTblGen", description=( - "Utility script for generating sphinx restructureText " - "documentation for command-line options from JSON-dump of " - "command-line options TableGen file." + "Utility script for generating Sphinx Markdown documentation for " + "command-line options from JSON-dump of command-line options " + "TableGen file." ), ) @@ -253,8 +246,8 @@ def create_argparser(): "--supplement", metavar="dir", help=( - "Directory containing supplemental RST documentation for options. " - "Each file must be named .rst.in (without any prefix); " + "Directory containing supplemental Markdown documentation for options. " + "Each file must be named .md.in (without any prefix); " "its contents are appended after the auto-generated HelpText." ), ) @@ -280,9 +273,9 @@ def remove_groups(groups, skip_options_groups): if args.supplement: supplement_dir = args.supplement for filename in os.listdir(supplement_dir): - if not filename.endswith(".rst.in"): + if not filename.endswith(".md.in"): continue - option_name = filename[: -len(".rst.in")] + option_name = filename[: -len(".md.in")] filepath = os.path.join(supplement_dir, filename) with open(filepath) as f: supplements[option_name] = f.read() diff --git a/docs/userguide/conf.py.in b/docs/userguide/conf.py.in index cfb8e276b..d58aff204 100644 --- a/docs/userguide/conf.py.in +++ b/docs/userguide/conf.py.in @@ -42,14 +42,21 @@ if _BREATHE_XML_DIR: templates_path = ["_templates"] -exclude_patterns = ["_build", "_build_*", "**/_build", "**/_build_*"] +exclude_patterns = [ + "_build", + "_build_*", + "**/_build", + "**/_build_*", + "documentation/options/*LinkerOptions.md", +] source_suffix = { - ".rst": "restructuredtext", ".md": "markdown", } master_doc = "index" +myst_enable_extensions = ["colon_fence", "deflist"] + language = "en" pygments_style = "sphinx" todo_include_todos = True diff --git a/docs/userguide/documentation/diagnostic_reports.md b/docs/userguide/documentation/diagnostic_reports.md new file mode 100644 index 000000000..c1fb812d0 --- /dev/null +++ b/docs/userguide/documentation/diagnostic_reports.md @@ -0,0 +1,197 @@ +# Diagnostic Reports + +**Linker diagnostics** refer to the messages and reports generated by a linker to help developers identify and resolve issues during the linking phase of compilation. These diagnostics can include: + +- Warnings and errors about malformed or unsupported linker scripts +- Misplaced or overlapping sections +- Unresolved symbols +- Alignment issues +- Plugin-related inconsistencies + +## What `-Wall` and `-Werror` Mean for ELD + +### `-Wall`: Enable All Warnings + +In the eld linker, `-Wall` enables a broad set of diagnostic warnings during the linking process. These warnings help developers catch: + +- Misuse of linker script commands +- Compatibility issues during linking +- Linker script forward references +- Suspicious section overlaps or misalignments +- Deprecated or unsupported syntax +- Potentially undefined behavior in symbol resolution +- Command line usage + +This flag is particularly useful in embedded development, where layout precision and deterministic behavior are critical. + +### `-Werror`: Treat Warnings as Errors + +When `-Werror` is used, any warning promoted by `-Wall` (or other `-W` flags) is treated as a **fatal error**, halting the linking process. This ensures: + +- No warnings are ignored during CI/CD builds +- Developers are forced to resolve all issues before producing a final binary +- Consistency across builds, especially when linker behavior may vary across toolchain versions + +This is especially important in embedded workflows, where eld is used to build critical components like firmware and device drivers. + +### Why It Matters in ELD + +- **Embedded Safety**: In embedded systems, even minor layout issues can lead to runtime faults. `-Wall -Werror` ensures these are caught early. +- **Plugin Infrastructure**: ELD supports linker plugins. These can emit custom diagnostics, and `-Wall` ensures they’re surfaced; `-Werror` enforces them. +- **Script Compatibility**: ELD aims for GNU linker script compatibility. These flags help validate script correctness during migration from GNU ld. + +### Example Usage + +```bash +eld -T script.ld -o firmware.elf main.o -Wall -Werror +``` + +This command will: + +- Use `script.ld` for layout +- Link `main.o` into `firmware.elf` +- Emit all warnings (`-Wall`) +- Fail the build if any warning is encountered (`-Werror`) + +## ELD Linker Warning Flags + +This document provides a categorized list of warning flags supported by the ELD linker. + +### General Warning Flags + +```{eval-rst} +.. list-table:: + :header-rows: 1 + + * - Flag + - Description + * - ``-Wall`` + - Enables all standard warnings supported by ELD. Useful for catching script and layout issues early. + * - ``-Warchive-file`` + - Warns about issues related to archive (.a) when they contain duplicate members. + * - ``-Wattribute-mix`` + - Flags inconsistent or conflicting section attributes across input files. + * - ``-Wbad-dot-assignments`` + - Warns when the ``.`` (location counter) is used in a way that may cause layout issues or undefined behavior. + * - ``-Wcommand-line`` + - Enables warnings for malformed or conflicting command-line options. + * - ``-Werror`` + - Treats all warnings as errors, halting the link process. + * - ``-Wlinker-script`` + - Enables warnings specific to linker script issues, such as malformed directives, deprecated syntax, or unsupported constructs. + * - ``-Wlinker-script-memory`` + - Focuses on memory region definitions in linker scripts, such as overlaps or undefined regions. + * - ``-Wwhole-archive`` + - Warns when ``--whole-archive`` is used inappropriately or causes symbol bloat. + * - ``-Wzero-sized-sections`` + - Flags sections that are defined but have zero size, which may indicate a script or input error. + * - ``-Wosabi`` + - Generates a warning when an input file's OS/ABI value differs from those encountered in previously processed files. +``` + +### Suppression Flags + +```{eval-rst} +.. list-table:: + :header-rows: 1 + + * - Flag + - Description + * - ``-Wno-archive-file`` + - Suppresses archive file-related warnings. + * - ``-Wno-attribute-mix`` + - Suppresses attribute mismatch warnings. + * - ``-Wno-error`` + - Allows warnings to be emitted without halting the link process. + * - ``-Wno-linker-script`` + - Disables linker script-related warnings. + * - ``-Wno-osabi`` + - Disables OS/ABI warnings. + * - ``-Wno-whole-archive`` + - Disables whole archive warnings. +``` + +## Archive Member Reports + +In addition to map-file text output, `--archive-member-report` emits +a JSON report with one record per archive-member inclusion event. This report +is generated even when `-Map` is not requested. + +Example linker invocation: + +``` +clang hw.o -Wl,--archive-member-report,report.json --ld-path=ld.eld +``` + +Schema (per `ArchiveMembers` entry): + +- `MemberPath`: Decorated member path such as `libc.a(malloc.o)`. +- `Archive` / `Member`: Archive and member split fields for archive members. +- `ReferrerObjectFile`: Requester when an object file caused the pull-in. +- `ReferrerArchive` / `ReferrerMember`: Requester when another archive + member caused the pull-in. +- `Symbol`: Symbol that triggered inclusion for non-whole-archive pulls. +- `WholeArchive` and `Reason`: Present when `--whole-archive` caused + inclusion (for example `Reason: --whole-archive`). +- `MemberReferencedSymbols`: Symbols referenced by the included member; this + enables tracing symbol chains such as `sys_init_tls -> malloc`. +- `MemberSymbols`: Symbols defined by the included member. +- `MemberIsArchiveMember` / `ReferrerIsArchiveMember`: Explicit booleans + describing whether each side of the edge is an archive member. +- `MemberIsBitcode` / `ReferrerIsBitcode`: Bitcode markers for mixed + native/bitcode links. + +Behavior notes: + +- The report is emitted even when map files are not requested. +- Both regular archives and thin archives are supported. +- For `--whole-archive` inclusion, `WholeArchive` is `true` and + `Reason` is emitted as `--whole-archive`. + +Example JSON snippet: + +``` +{ + "ArchiveMembers": [ + { + "MemberPath": "libc.a(malloc.o)", + "Archive": "libc.a", + "Member": "malloc.o", + "ReferrerArchive": "libstandalone.a", + "ReferrerMember": "sys_init_tls.o", + "Symbol": "malloc", + "MemberReferencedSymbols": ["sbrk", "errno"] + } + ] +} +``` + +### Tracing behavior with examples + +Use `utils/archive_member_report/archive_member_trace.py` to understand *why* members were +included and to follow chains through archive members. + +Examples: + +``` +# Why was malloc needed? +python3 utils/archive_member_report/archive_member_trace.py report.json --symbol malloc --why-needed + +# Trace all members matching a glob. +python3 utils/archive_member_report/archive_member_trace.py report.json --pattern "*libc.a(*malloc*.o)" + +# Show referenced symbols for matching archives. +python3 utils/archive_member_report/archive_member_trace.py report.json --referenced-symbols-archive "*libc.a" + +# Show defined symbols for matching members. +python3 utils/archive_member_report/archive_member_trace.py report.json --defined-symbols-member "*tls*.o" + +# Thin archive works the same way. +clang main.o -Wl,--archive-member-report,report.json thinlib.a +python3 utils/archive_member_report/archive_member_trace.py report.json --symbol foo +``` + +The `--symbol` and `--pattern` options are mutually exclusive. For +`--why-needed`, the output highlights the requesting file/member and prints +a readable chain so each pull-in reason is visible as a separate trace. + diff --git a/docs/userguide/documentation/diagnostic_reports.rst b/docs/userguide/documentation/diagnostic_reports.rst deleted file mode 100644 index 65b35e392..000000000 --- a/docs/userguide/documentation/diagnostic_reports.rst +++ /dev/null @@ -1,197 +0,0 @@ -Diagnostic Reports -################## - -**Linker diagnostics** refer to the messages and reports generated by a linker to help developers identify and resolve issues during the linking phase of compilation. These diagnostics can include: - -- Warnings and errors about malformed or unsupported linker scripts -- Misplaced or overlapping sections -- Unresolved symbols -- Alignment issues -- Plugin-related inconsistencies - -What ``-Wall`` and ``-Werror`` Mean for ELD -============================================================== - -``-Wall``: Enable All Warnings ------------------------------- - -In the eld linker, ``-Wall`` enables a broad set of diagnostic warnings during the linking process. These warnings help developers catch: - -- Misuse of linker script commands -- Compatibility issues during linking -- Linker script forward references -- Suspicious section overlaps or misalignments -- Deprecated or unsupported syntax -- Potentially undefined behavior in symbol resolution -- Command line usage - -This flag is particularly useful in embedded development, where layout precision and deterministic behavior are critical. - -``-Werror``: Treat Warnings as Errors -------------------------------------- - -When ``-Werror`` is used, any warning promoted by ``-Wall`` (or other ``-W`` flags) is treated as a **fatal error**, halting the linking process. This ensures: - -- No warnings are ignored during CI/CD builds -- Developers are forced to resolve all issues before producing a final binary -- Consistency across builds, especially when linker behavior may vary across toolchain versions - -This is especially important in embedded workflows, where eld is used to build critical components like firmware and device drivers. - -Why It Matters in ELD ----------------------- - -- **Embedded Safety**: In embedded systems, even minor layout issues can lead to runtime faults. ``-Wall -Werror`` ensures these are caught early. -- **Plugin Infrastructure**: ELD supports linker plugins. These can emit custom diagnostics, and ``-Wall`` ensures they’re surfaced; ``-Werror`` enforces them. -- **Script Compatibility**: ELD aims for GNU linker script compatibility. These flags help validate script correctness during migration from GNU ld. - -Example Usage -------------- - -.. code-block:: bash - - eld -T script.ld -o firmware.elf main.o -Wall -Werror - -This command will: - -- Use ``script.ld`` for layout -- Link ``main.o`` into ``firmware.elf`` -- Emit all warnings (``-Wall``) -- Fail the build if any warning is encountered (``-Werror``) - -ELD Linker Warning Flags -======================== - -This document provides a categorized list of warning flags supported by the ELD linker. - -General Warning Flags ----------------------- - -.. list-table:: - :header-rows: 1 - - * - Flag - - Description - * - ``-Wall`` - - Enables all standard warnings supported by ELD. Useful for catching script and layout issues early. - * - ``-Warchive-file`` - - Warns about issues related to archive (.a) when they contain duplicate members. - * - ``-Wattribute-mix`` - - Flags inconsistent or conflicting section attributes across input files. - * - ``-Wbad-dot-assignments`` - - Warns when the ``.`` (location counter) is used in a way that may cause layout issues or undefined behavior. - * - ``-Wcommand-line`` - - Enables warnings for malformed or conflicting command-line options. - * - ``-Werror`` - - Treats all warnings as errors, halting the link process. - * - ``-Wlinker-script`` - - Enables warnings specific to linker script issues, such as malformed directives, deprecated syntax, or unsupported constructs. - * - ``-Wlinker-script-memory`` - - Focuses on memory region definitions in linker scripts, such as overlaps or undefined regions. - * - ``-Wwhole-archive`` - - Warns when ``--whole-archive`` is used inappropriately or causes symbol bloat. - * - ``-Wzero-sized-sections`` - - Flags sections that are defined but have zero size, which may indicate a script or input error. - * - ``-Wosabi`` - - Generates a warning when an input file's OS/ABI value differs from those encountered in previously processed files. - -Suppression Flags ------------------- - -.. list-table:: - :header-rows: 1 - - * - Flag - - Description - * - ``-Wno-archive-file`` - - Suppresses archive file-related warnings. - * - ``-Wno-attribute-mix`` - - Suppresses attribute mismatch warnings. - * - ``-Wno-error`` - - Allows warnings to be emitted without halting the link process. - * - ``-Wno-linker-script`` - - Disables linker script-related warnings. - * - ``-Wno-osabi`` - - Disables OS/ABI warnings. - * - ``-Wno-whole-archive`` - - Disables whole archive warnings. - -Archive Member Reports -====================== - -In addition to map-file text output, ``--archive-member-report`` emits -a JSON report with one record per archive-member inclusion event. This report -is generated even when ``-Map`` is not requested. - -Example linker invocation:: - - clang hw.o -Wl,--archive-member-report,report.json --ld-path=ld.eld - -Schema (per ``ArchiveMembers`` entry): - -- ``MemberPath``: Decorated member path such as ``libc.a(malloc.o)``. -- ``Archive`` / ``Member``: Archive and member split fields for archive members. -- ``ReferrerObjectFile``: Requester when an object file caused the pull-in. -- ``ReferrerArchive`` / ``ReferrerMember``: Requester when another archive - member caused the pull-in. -- ``Symbol``: Symbol that triggered inclusion for non-whole-archive pulls. -- ``WholeArchive`` and ``Reason``: Present when ``--whole-archive`` caused - inclusion (for example ``Reason: --whole-archive``). -- ``MemberReferencedSymbols``: Symbols referenced by the included member; this - enables tracing symbol chains such as ``sys_init_tls -> malloc``. -- ``MemberSymbols``: Symbols defined by the included member. -- ``MemberIsArchiveMember`` / ``ReferrerIsArchiveMember``: Explicit booleans - describing whether each side of the edge is an archive member. -- ``MemberIsBitcode`` / ``ReferrerIsBitcode``: Bitcode markers for mixed - native/bitcode links. - -Behavior notes: - -- The report is emitted even when map files are not requested. -- Both regular archives and thin archives are supported. -- For ``--whole-archive`` inclusion, ``WholeArchive`` is ``true`` and - ``Reason`` is emitted as ``--whole-archive``. - -Example JSON snippet:: - - { - "ArchiveMembers": [ - { - "MemberPath": "libc.a(malloc.o)", - "Archive": "libc.a", - "Member": "malloc.o", - "ReferrerArchive": "libstandalone.a", - "ReferrerMember": "sys_init_tls.o", - "Symbol": "malloc", - "MemberReferencedSymbols": ["sbrk", "errno"] - } - ] - } - -Tracing behavior with examples ------------------------------- - -Use ``utils/archive_member_report/archive_member_trace.py`` to understand *why* members were -included and to follow chains through archive members. - -Examples:: - - # Why was malloc needed? - python3 utils/archive_member_report/archive_member_trace.py report.json --symbol malloc --why-needed - - # Trace all members matching a glob. - python3 utils/archive_member_report/archive_member_trace.py report.json --pattern "*libc.a(*malloc*.o)" - - # Show referenced symbols for matching archives. - python3 utils/archive_member_report/archive_member_trace.py report.json --referenced-symbols-archive "*libc.a" - - # Show defined symbols for matching members. - python3 utils/archive_member_report/archive_member_trace.py report.json --defined-symbols-member "*tls*.o" - - # Thin archive works the same way. - clang main.o -Wl,--archive-member-report,report.json thinlib.a - python3 utils/archive_member_report/archive_member_trace.py report.json --symbol foo - -The ``--symbol`` and ``--pattern`` options are mutually exclusive. For -``--why-needed``, the output highlights the requesting file/member and prints -a readable chain so each pull-in reason is visible as a separate trace. diff --git a/docs/userguide/documentation/editor_support.rst b/docs/userguide/documentation/editor_support.md similarity index 54% rename from docs/userguide/documentation/editor_support.rst rename to docs/userguide/documentation/editor_support.md index fbafd8cb5..bf2d4f660 100644 --- a/docs/userguide/documentation/editor_support.rst +++ b/docs/userguide/documentation/editor_support.md @@ -1,24 +1,22 @@ -Editor Support -============== +# Editor Support -.. contents:: - :local: +```{contents} +:local: true +``` -Overview --------- +## Overview ELD provides editor integration files that add syntax highlighting for ELD -linker scripts and map files. The integration is currently available for +linker scripts and map files. The integration is currently available for **Vim** (version 9.1 or later). -Vim Syntax Highlighting ------------------------ +## Vim Syntax Highlighting -Supported File Types -~~~~~~~~~~~~~~~~~~~~ +### Supported File Types The filetype-detection file recognises the following files automatically: +```{eval-rst} .. list-table:: :header-rows: 1 :widths: 30 70 @@ -29,25 +27,26 @@ The filetype-detection file recognises the following files automatically: - ELD linker scripts * - ``*.map`` (first line matches ``# Linker``) - ELD map files +``` To set the filetype manually inside an open Vim buffer: -.. code-block:: vim +```vim +:set filetype=eld +``` - :set filetype=eld - -Installation -~~~~~~~~~~~~ +### Installation Copy the two Vim files into your Vim runtime directories: -.. code-block:: bash - - cp etc/vim/eld.vim ~/.vim/syntax/eld.vim - cp etc/vim/ftdetect_eld.vim ~/.vim/ftdetect/eld.vim +```bash +cp etc/vim/eld.vim ~/.vim/syntax/eld.vim +cp etc/vim/ftdetect_eld.vim ~/.vim/ftdetect/eld.vim +``` -Vim resolves syntax files by filetype name (``filetype=eld`` → -``syntax/eld.vim``), so the filenames must be kept exactly as shown. +Vim resolves syntax files by filetype name (`filetype=eld` → +`syntax/eld.vim`), so the filenames must be kept exactly as shown. After copying the files, open any linker script or map file and Vim will apply syntax highlighting automatically. + diff --git a/docs/userguide/documentation/eld_debugging_guide.md b/docs/userguide/documentation/eld_debugging_guide.md new file mode 100644 index 000000000..8ad0b48e2 --- /dev/null +++ b/docs/userguide/documentation/eld_debugging_guide.md @@ -0,0 +1,1266 @@ +# ELD (debugging guide) + +This document describes the high-level flow of how ELD executes a link, with +call-site pointers and practical tips for debugging failures. + +```{contents} +:local: true +``` + +## Big picture + +At a high level, a link invocation looks like this: + +1. **eld main** expands response files and selects a driver flavor/target. +2. **Driver** parses options and builds an ordered list of input actions. +3. **Linker prepare** initializes target/emulation, inputs, and plugins, then + reads and normalizes input files (and may run LTO). +4. **Linker link** resolves symbols/relocations, lays out output sections, then + emits the final ELF and optional map files. +5. **Optional diagnostics**: reproduce tarball/mapping file, plugin activity + log, timing stats, summary, etc. + +## Entry point and driver selection + +The executable entry point is `tools/eld/eld.cpp`: + +- Expands `@response` files via `llvm::cl::ExpandResponseFiles(...)`. +- Creates a `Driver` and calls + `Driver::setDriverFlavorAndInferredArchFromLinkCommand(...)`. +- Creates a GNU-ld-compatible driver (`GnuLdDriver`) and calls + `GnuLdDriver::link(...)`. + +Driver flavor selection is implemented in `lib/LinkerWrapper/Driver.cpp`: + +- First tries to infer a target from the program name (e.g. `arm-link`, + `aarch64-link`, `hexagon-link`). +- Otherwise inspects early arguments like `-m ` or `-march` to + select a target-specific driver. + +Environment hooks that affect arguments: + +- `ELDFLAGS`: appended to the link command by the driver (useful for always-on + debug flags). + +## Argument parsing and preprocessing + +The top-level flow of option parsing and dispatch is in +`lib/LinkerWrapper/GnuLdDriver.cpp` (`GnuLdDriver::link(...)`): + +1. `parseOptions(...)` +2. `processLLVMOptions(...)` (parses `-mllvm ...` arguments) +3. `processTargetOptions(...)` (handles `-mtriple`, `-march`, `-mabi`, + `-m `, etc.) +4. `processOptions(...)` (general linker options) +5. `checkOptions(...)` and `overrideOptions(...)` +6. `createInputActions(...)` to build the ordered action list +7. `doLink(...)` to run the actual link pipeline + +If you suspect argument/option issues, start with: + +- `--verbose` (or `--verbose=`) +- `--trace=command-line` +- `--trace=files` or `-t` (prints processed files) +- `--error-style=GNU` or `--error-style=LLVM` (if output formatting matters) + +## Input actions (what gets fed to the linker) + +After parsing options, the driver builds a sequence of actions that are later +"activated" to create inputs: + +- `-T