Snippet Scanners¶

The scanner key of a target rule or a source rule settles how the regions of a file that the rule owns are found:

markers is the default and is what a project normally wants, because a marker tells a reader of the target file what is going on. Reach for whole when an entire file is one copy of one snippet, so that there is nothing in it to mark and nothing to match. Reach for regex when a file cannot carry a comment where the region belongs, or when it already has a shape worth matching. Reach for json when the file is JSON, because it does not support marker comments and regular expressions would be brittle.

The Whole File¶

A rule with scanner = "whole" owns everything its files hold, so that one file is one copy of one snippet:

[[targets]]
patterns = ["licenses/*.txt"]
scanner = "whole"
snippets = ["notice"]

The region of such a rule runs from the first character of the file to the last, which leaves nothing to mark and nothing to search for, so the scan of a whole file cannot fail. The region carries no name of its own, exactly as the match of an expression without a name group does, so snippets names the one snippet every file of the rule holds and has to name exactly one.

Nothing is laid out around the result of the template, so a file ends up holding exactly what the render of the rule produces. A snippet carries the line ending of its last line, which a file receiving it keeps, and a file that is empty holds an empty copy rather than no region at all.

The rule above keeps its files identical without a definition in snipwise.md, because a whole file gives its snippet back and Which Copy Wins then settles which file holds the original. The same rule written as [[sources]] says outright which file that is, and is how a file becomes the original of a snippet that marked blocks elsewhere receive.

A whole rule owns every character of a file, so no other rule ever owns a region in the same file. A rule that finds one there and holds a different snippet is refused as an overlap. A rule that holds the same snippet is settled by Multiple Rules instead, where a whole rule counts as narrowed, because it names the one snippet it holds.

Markers¶

A file selected by a rule with scanner = "markers", which is the default, receives a snippet by containing a begin and an end marker for it. Snipwise replaces all lines strictly between the two markers, and never touches the marker lines themselves. A marker starts with the name of the configuration file, snipwise.md, so that whoever runs into it knows which file to edit instead.

Snipwise does not parse the target file and has no notion of file formats. It only looks for lines that contain the marker text, so the comment syntax of any language works out of the box:

The rest of the marker line is yours, so you can add a remark for other readers:

# snipwise.md BEGIN python-versions (do not edit here)

The indentation of the begin marker is applied to every inserted line, which lets you place a snippet inside an indented block. A file may contain the same snippet more than once.

The marker text is matched literally, including its case, so a line holding SNIPWISE.md BEGIN is not a marker and passes unnoticed.

An Expression of Your Own¶

A rule with scanner = "regex" finds its regions with the expression in its regex key, written in the syntax of the re module. Every match of that expression is a region, and the file needs no marker at all:

[[targets]]
patterns = ["paper/*.tex"]
scanner = "regex"
regex = '\\newcommand\{\\snipwise(?P<name>\w+)\}\{(?P<content>.*?)\}'
render = "{{ content | unwrap }}"

This rule keeps a LaTeX file in step through its macro definitions, so a line such as the following receives the snippet named tagline:

\newcommand{\snipwisetagline}{Keep the copies of a text in step.}

Two named groups have a meaning:

• content (required) covers the characters that Snipwise replaces. Everything else the expression matches is left alone, so the expression is free to match far more than it hands over.
• name names the snippet the region holds. A rule whose expression has no name group names its snippet in snippets instead, which is what suits a file whose shape says nothing about which snippet it holds:

[[targets]]
patterns = ["src/mypackage/__init__.py"]
scanner = "regex"
regex = '__version__ = "(?P<content>[^"]*)"'
snippets = ["version"]
render = "{{ content | unwrap }}"

Such a rule claims exactly the one snippet it names, so it may share a file with a rule that receives every other snippet of it. An expression that does have a name group leaves snippets its usual meaning, which is the narrowing described in Multiple Rules.

A region of this scanner is a span of characters and not a block of lines, so nothing is laid out around what the template produces, and a trailing newline that the snippet carries ends up inside the span. That is why both rules above render with unwrap, which joins the paragraph of the snippet into one line and drops its final newline: without it, the closing brace of the macro would end up on a line of its own. A region that does cover whole lines, such as the block of comment lines below, wants the newline and therefore renders without it.

Write the value of regex as a TOML literal string in single quotes, so that the backslashes of the expression reach Snipwise as they are written. Flags belong inline in the expression, as in (?m)^version: (?P<content>.*)$, because Snipwise sets none of them itself.

Every other named group of the expression reaches the template of the rule under its own name, which is what lets a replacement depend on what was matched:

[[targets]]
patterns = ["src/**/*.py"]
scanner = "regex"
regex = '(?m)^(?P<indent>[ \t]*)# (?P<name>[\w.-]+):\n(?P<content>(?:^[ \t]*#(?! [\w.-]+:$).*\n)*)'
render = "{{ content | prefix('# ') | prefix(indent) }}"

The lookahead is what stops the content group at the header of the next block. Without it, a header that directly follows a run of comment lines is swallowed by that run, and the snippet it names loses its region without a word of complaint.

A group that did not take part in a match is absent rather than empty, so a template that names it fails instead of silently rendering nothing.

Two mistakes are worth knowing about in advance:

• An expression that matches more of a file than you meant will replace more of a file than you meant. There is no end marker to bound the damage, so prefer an anchored expression and a non-greedy content group, and run snipwise check --diff before snipwise fix.
• An expression that Snipwise cannot use is refused while snipwise.md is read, before any file is touched. That covers one that does not compile, one without a content group, and one without a name group whose rule does not name exactly one snippet.

Values of a JSON File¶

A JSON file carries no comment, so a marker has nowhere to go, and the shape of such a file says nothing about which snippet a value holds. A rule with scanner = "json" is therefore told what to write where, in a list of insert entries:

[[targets]]
patterns = ["package.json"]
scanner = "json"
insert = [
  { snippet = "tagline", pointer = "/description", render = "{{ content | unwrap }}" },
  { snippet = "keywords", pointer = "/keywords", shape = "lines" },
]

Every entry has two required keys and two optional ones:

Required:

• snippet is the name of the snippet that this value holds. A JSON file has nothing to name it with, so the rule names it.
• pointer is an RFC 6901 pointer addressing the value: /description is a member of the object at the root, /engines/node a member of a nested object and /keywords/0 the first element of an array. Write ~1 for a slash inside a key and ~0 for a tilde, and write the empty string to address the whole document.

Optional:

• shape says how the rendered text becomes a JSON value. It is text by default.
• render is a template for this entry alone, which replaces the render of the rule. Entries that share a transformation can leave it to the rule.

The entries also say what the rule claims. A json rule is narrowed to the snippets its entries name, exactly as another rule is narrowed by its snippets key, so two json rules may address different values of one file, and a json rule may sit beside a rule that receives everything else in it. The snippets key itself is refused here, because a list repeating the entries could only fall out of step with them.

A pointer that does not resolve is an error that names the file and the pointer. Snipwise copies text and never creates the member or the element that would hold it, so the value has to be there already, whatever it currently holds.

The value is spliced as text and the document is never loaded and dumped again, which leaves the key order, the indentation and the whitespace of the rest of the file exactly as they were. Running snipwise fix on:

{
  "name": "demo",
  "description": "stale",
  "keywords": [
    "old"
  ],
  "private": true
}

changes the two addressed values and nothing else:

{
  "name": "demo",
  "description": "Keep the copies of a text in step.",
  "keywords": [
    "snippet",
    "consistency"
  ],
  "private": true
}

There are three ways to write a value:

text writes the rendered text as a single string, escapes and all. A snippet ends with a line ending, which such a string keeps as \n, so a rule writing one into a string usually renders with unwrap or trim.

lines writes one element per line. It takes the layout of the array it replaces: an array written over several lines is rewritten over several lines, with the indentation and the line endings its author gave it, while any other value gives a single line. Both of these therefore stay as they are written:

  "keywords": ["snippet", "consistency"],
  "files": [
    "dist",
    "README.md"
  ]

json takes the rendered text as a fragment of JSON. It is parsed, so that a broken template cannot leave a file that no longer is JSON, and it is then spliced exactly as the template wrote it, which leaves the layout to you. A fragment is any JSON value, including a bare number or string.

Two entries of one rule may not address the same value, and neither may one address a part of what another addresses, because the result would depend on the order in which the entries happen to be written. Both are refused while snipwise.md is read.

Snipwise reports a JSON value by its pointer rather than by the line it sits on:

package.json:/description: out-of-date snippet 'tagline'

Finally, the file has to be JSON and nothing else. Comments and trailing commas belong to other dialects, so a tsconfig.json that uses them is reported as malformed rather than repaired.