Changelog

v3.0.0a4

  • LineEndings no longer clobbers newly-written values to simfiles. Previously, it treated the original MSD parameter as canonical (it is not).

  • The strict parameter finally defaults to False as originally planned.

v3.0.0a3

  • msdparser was upgraded to version 3.0.0a7, which fixes some bugs related to comments & escape sequences.

  • tidy() presets now work; their behaviors were implemented, but the tidy() function did nothing with them. Whoops!

v3.0.0a2

New features

A new module, simfile.tidy, is now available. This module offers one function, tidy(), that acts as the interface to various behaviors which normalize a chart’s serialized representation. It also offers some presets that package these behaviors together.

Miscellaneous

  • Duplicate keys are now preserved, although they are not accessible through simfile / chart attributes. They are stored in the underlying OrderedDict with keys like PROP:n, where PROP is the uppercase property name and n is a 1-indexed count of how many duplicates have appeared so far. These keys are purely an implementation detail; aforementioned :n suffix is removed during serialization.

  • Key casing is now preserved during serialization. Note that keys are still coerced to uppercase for the OrderedDict interface. For example, if a simfile’s title is declared using #Title:MAX 300;, the title will be accessible via title and sim["TITLE"], but it will serialize back to disk with the same case seen during parsing.

  • Ephemeral details such as whitespace & comments are now preserved more intelligently than before. In particular, any comment above a chart definition is now considered attached to the chart; new simfile-level properties will appear above the comment, instead of below it.

v3.0.0a1

Breaking changes

Warning

simfile 3.0 introduces some breaking changes that you may need to update your code to handle:

Python 3.10 or higher required

simfile’s minimum Python version was raised from 3.6 to 3.10.

Strict parsing is now off by default

Functions in the top-level module, such as simfile.open(), simfile.load(), and simfile.mutate(), now default to strict=False. If you want strict parse errors, pass strict=True to restore the old behavior.

Features that are now optional

The simfile.assets submodule now requires installing simfile with the assets extra. Run one of these commands to add it to your project:

pip install 'simfile[assets]'  # or...
poetry add 'simfile[assets]'  # or...
rye add simfile --features assets

The filesystem parameter of various functions & methods now requires installing simfile with the fs extra. Run one of these commands to add it to your project:

pip install 'simfile[fs]'  # or...
poetry add 'simfile[fs]'  # or...
rye add simfile --features fs

If you want to use both of these optional features, join them with a comma:

pip install 'simfile[assets,fs]'  # or...
poetry add 'simfile[assets,fs]'  # or...
rye add simfile --features assets,fs

Which command to use depends on what package manager you’re using for your project. If you’re unsure, use the pip command.

Some dict operations no longer supported on simfiles & charts

The Simfile and Chart classes (both SM and SSC) no longer inherit OrderedDict; instead, they have a private OrderedDict attribute and forward common dictionary methods & operations to it.

These operations that were previously inherited by Simfile and Chart are not forwarded:

  • The clear, copy, popitem, reversed, setdefault, and update methods from dict are no longer supported.

  • The | and |= operators are no longer supported.

These operations are forwarded and should behave the same as before:

  • [key] indexing & assignment

  • len() and iteration

  • The move_to_end method from OrderedDict

  • The keys, values, items, get, and pop methods from dict

TimingData and displaybpm() parameters changed

TimingData and displaybpm() both now take a single argument, a Simfile or an AttachedChart (a chart taken from a Simfile - more on this below). Previously, they took one or two arguments, a required Simfile and an optional Chart.

If your code passed both a Simfile and a Chart to either of these classes/functions, you can probably fix it by removing the first argument (the simfile). However, this might fail if your Chart is not an AttachedChart. In that case, append your chart to the simfile’s charts to attach it to the simfile. Then you can read the attached chart back from the simfile’s charts.

group_notes() parameters changed

group_notes()’ optional include_note_types parameter was removed. Use the built-in filter function on the input note data instead.

It was unclear how this parameter interacted with the other parameters. The fact that such a simple operation was included as a parameter suggested that it would do something “smarter” under the hood. Consequently, it was very easy to make mistakes like excluding NoteType.TAIL while setting join_heads_to_tails to True. Other submodules of simfile that used group_notes() under the hood even made mistakes like these!

Enhancements

Strict or non-strict parsing is now pervasive

Simfiles and charts now know whether they were opened with strict parsing (or default to False when created in memory). When a simfile is parsed with strict=False:

  • Stray text and missing semicolon recovery are both allowed during MSD parsing.

  • BeatValues uses a lenient float parser that permits trailing junk data.

Attached charts

All charts under a simfile’s charts attribute are now instances of AttachedChart, which is a type union for AttachedSMChart and AttachedSSCChart. These are subclasses of SMChart and SSCChart that store a reference to the simfile they came from.

Simfile (de)serialization is now exact

Deserializing and serializing a Simfile is now byte-for-byte symmetric in most cases. For example, if you open a simfile with simfile.mutate() and don’t make any changes, the output file should exactly match the input file. This includes whitespace, comments, duplicate & lowercase keys, and any other ephemeral details.

Note

Known exceptions: Non-chart keys found after the first chart will be moved before the first chart. This is unlikely to change.

Similarly, newly created properties on Simfile objects now use a heuristic to determine their suffix (the ; and any following whitespace) based on the initial properties seen in the simfile. For example, a newly-added property on a file containing Windows-style newlines will have its suffix set to ;\r\n to match the rest of the file.

msdparser upgraded

simfile’s dependency on msdparser was bumped to 3.0. This is what made the byte-for-byte symmetry described above possible. It also fixes a few parsing bugs that are described below.

Bugfixes

Note counts match StepMania

The simfile.notes.count module has been replaced by simfile.notes.counter, which omits fake notes from counts. The counter module contains many functions with the same names, but which take an AttachedChart instead of NoteData. This fixes a long-standing misparity with StepMania’s counts.

Additionally, count_hands() now takes active holds and rolls into account. This fixes hands being undercounted when compared to StepMania.

Missing semicolon recovery matches StepMania

Missing semicolon recovery is a feature of StepMania’s parser that checks for new lines starting with # and treats them as new parameters, even if there was no preceding ;. This was implemented in the msdparser 2.0 dependency, but it didn’t quite have parity with StepMania’s parser. In particular, the new line wasn’t allowed to have whitespace before the #; only the two-character sequence \n# could trigger it. msdparser 3.0 corrects this so that preceding whitespace is allowed. It also correctly removes all trailing whitespace (such as the line break) from the preceding component, which was another mismatch between msdparser 2.0 and StepMania.

openpack() passes kwargs to open()

It didn’t before. Whoops!

v2.1.1

Bugfixes

Two bugs in simfile 2.1.0’s SSC implementation broke multi-value properties, causing them to be truncated or mangled past the first value. This release fixes these issues:

  1. When opening an SSC file, the DISPLAYBPM and ATTACKS properties of both simfiles and charts no longer stop parsing at the first :. For DISPLAYBPM, this meant a BPM range of 120:240 would have been incorrectly parsed as a static BPM of 120. ATTACKS were completely broken as they use colon as a separator.

  2. The aforementioned properties are now correctly serialized from SSCChart; previously, they would have been escaped with backslashes. This bug had the same effects described above, but only affected manual assignment of multi-value properties (e.g. chart.displaybpm = "120:240") since the first bug shadowed this bug during deserialization.

v2.1.0

New features

  • The new simfile.dir module offers SimfileDirectory and SimfilePack classes for nagivating simfile filesystem structures.

  • The new simfile.assets module provides an Assets class that can reliably discover paths to simfile assets, even if they’re not specified in the simfile.

  • The top-level simfile module now offers opendir() and openpack() functions as simplified interfaces to the simfile.dir API.

  • PyFilesystem2 has been integrated throughout this library’s filesystem interactions, enabling OS and non-OS filesystems to be traversed using the same code. All functions, methods, and constructors that lead to filesystem interactions now have an optional filesystem parameter for specifying a PyFS filesystem object. When omitted, the filesystem defaults to the native OS filesystem as before.

  • The DisplayBPM classes now all expose the same four properties; the ones that don’t apply to a particular class return None. This enables you to handle all three cases without having to import the types for isinstance checks. Refer to Getting the displayed BPM for more details.

Bugfixes

  • The charts property on simfiles is now writable, meaning the list of charts can be overwritten directly (not just added to / removed from).

  • Backslash escape sequences and multi-value MSD parameters are now handled correctly, both when opening and serializing simfiles. See the Enhancements section below for more details.

  • sm_to_ssc() no longer produces invalid output when there are negative BPMs or stops in the timing data. (It throws NotImplementedError as a temporary stopgap. In the future, negative timing data will be converted to warps, as StepMania does automatically.)

  • Various type annotations have been improved throughout the library. In particular, Iterator input arguments have been replaced with Iterable so that you don’t need to wrap them in iter(...) to suppress type errors from static analyzers.

Enhancements

  • The dependency on msdparser has been upgraded to version 2. This corrects parsing of escape sequences and multi-value parameters, meaning that : and \ characters inside a value are handled the same way as in StepMania. Additionally, parsing is now up to 10 times faster than before!

v2.0.1

Bugfix: The dependency on msdparser 1.0.0 was mis-specified in both the Pipfile and setup.py. Publishing msdparser 2.0.0-beta.3 (a breaking release) caused fresh installs to be broken. This patch fixes the version specification in both files.

v2.0.0

Initial stable release of version 2. Refer to Migrating from simfile 1.0 to 2.0 for a general overview of the changes since version 1.