Skip to content

feat: self-similarity visualizer #526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 41 commits into
base: ui2
Choose a base branch
from

Conversation

katestange
Copy link
Member

By submitting this PR, I am indicating to the Numberscope maintainers that I have read and understood the contributing guidelines and that this PR follows those guidelines to the best of my knowledge. I have also read the pull request checklist and followed the instructions therein.


This PR adds a self-similarity visualizer to replace ShiftCompare.

In addition, it adds biggcd, valuation and divides to math.ts. These may not be needed once math.js does all its bigint stuff. It also adds a few more unit tests for modulo.

Things still to be done:

  • Finalize parameters interaction
  • Add documentation
  • Update snapshots

mbavec and others added 30 commits June 2, 2024 15:01
Adds a route that shows a panel of thumbnails of multiple different specimens. This can be used, for example, for a curated list of interesting visualizers, or for an individual user's own visualizers.
---------
Co-authored-by: Max Derbenwick <[email protected]>
Co-authored-by: maxd0328 <[email protected]>
Co-authored-by: Kaldis Berzins <[email protected]>
Co-authored-by: mbavec <[email protected]>
* Default implementation of aspect ratio in P5 visualizer template
* Responsive parameters displayed together with visualization
* New graphic design
* New typing system for parameters
* Drag and drop with tiling and windows
* Fixed name and description properties
* Setup specimen structure
* Moved the drag and drop into scope
(and multiple other small changes/fixes, too numerous to enumerate here)
---------
Co-authored-by: Max Derbenwick <[email protected]>
Co-authored-by: maxd0328 <[email protected]>
Co-authored-by: Kaldis Berzins <[email protected]>
Co-authored-by: ZiggyBeijer <[email protected]>
Co-authored-by: Glen Whitney <[email protected]>
Cause the URL of the Specimen Editor to encode the specimen name, sequence, and visualizer and all of their
parameters. Conversely, when visiting a URL with these properties encoded, decode them and re-create the
corresponding Speciment in the Editor.
---------

Co-authored-by: Max Derbenwick <[email protected]>
Co-authored-by: maxd0328 <[email protected]>
Co-authored-by: Kaldis Berzins <[email protected]>
Co-authored-by: ZiggyBeijer <[email protected]>
Co-authored-by: Glen Whitney <[email protected]>
* refactor: Add a new unused browserCaching system
Originally authored by @kaldis-berzins et. al.
* feat: resizing and URL integration

  Changes to tabs
    * Added buttons that let you control the tabs without dragging
    * Support minimizing of tabs.
    * Added a highlight to the tab that is currently selected.
    * Initial mobile compatibility in the style section of Tab.vue

  Resizing
    * Added a reset function to a specimen. This is a hard reset that is used
      when the specimen is resized and it doesn't override resized function.
    * Added a resized function to both specimens and visualizers. It is not
      required for visualizers as they can just not do anything and the
      specimen will hard reset them.

  Scope
    * Adds rudimentary mobile support.

Originally authored by @kaldis-berzins et al.

* fix: Dock back to where you were and non-overlapping initial tab positions

* doc: fix description of docking button

* refactor: Use consistent tab minimized height (and revert icon name change)
Navbar
  * Removes the navbar from App.vue. The navbar now has to be added
    to each view separately because it may be different for differrent views.
  * Makes the navbar taller and adds a slot to accommodate the specimen bar.

Specimen bar
  * Adds the specimen bar with replay, pause/resume, share, and save buttons.

Visualizers
  * Adds a continue function to be executed by the resume button.
* feat: Gallery integration

  Thumbnails
    * Adds thumbnails to gallery cards. They are essentially just specimens
      contained entirely within a little 200x200 container.
    * Thumbnails own their specimen, so the cards must retrieve the name
      of the specimen on their own.
        - Adds methods to Specimen.ts to enable this:
          getNameFromURL and getSequenceNameFromURL.

  Gallery
    * Has two sections: Saved specimens and Featured specimens.
    * Implements saving specimens and loading them from the gallery.
    * Featured specimens can be added to the list by editing
      shared/defineFeatured.ts. This file will automatically turn into cards
      on the gallery page.

  Scope
    * Current specimen is now saved in browser as well

* refactor: Merge SpecimenCard and FeaturedCard components

* style: Use 'pointer' cursor on specimen cards

* fix: Click anywhere on specimen card to load (and add featureds)

* fix: Numberscope logo goes to current specimen (and add featured)

* fix: Reduce leakage of visualizers and limit draw time in Gallery

* fix: Chaos allows no last element; P5Visualizer leave cursor be

* fix: Avoid filling in tentative values for optional parameters

* fix: Allow FactorHistogram to redraw on parameter changes

* fix: Display the fully-qualified sequence name in previews

* test: add missing browserCaching tests

* refactor: Improve Paramable API

  Prior to this commit, it was necessary in several places to manually
  validate, set isValid to true, and assignParameters. Now, whenever
  validate() is called, the isValid property is automatically updated, and
  if it happens that isValid is true, then assignParameters is
  automatically invoked.

  In addition, adds a validateIndividual() method to ParamableInterface
  that is needed by the ParamEditor. Including this method in Paramable
  itself eliminates previous code duplication between Paramable and
  ParamEditor.

* refactor: remove redundancy in defineFeatured

* fix: Specimen delete button in gallery should not open specimen
…umberscope#377)

Note that there remains an issue of the drop zone remaining visible when a Gallery specimen is loaded, but this will be addressed in the next PR.
* feat: Sequence and Visualizer switchers, first pass at cleaned

* fix: Always show the current background and stroke colors in Turtle

* fix: Don't refresh empty default parameters

* ui: move sequence search bar to right of Sequence Switcher popup title

* feat: Switchers now show thumbnails of the modified visualizers

  Implementing this feature involves abstracting the sections of the
  Gallery into a new SpecimensGallery component, and then using that
  component in the SwitcherModal. Note that in order to accommodate the
  desired captions of the SwitcherModal, the CardSpecimen interface
  used to specify the contents of a SpecimensGallery has to be generalized
  to allow arbitrary subtitles.

* fix: Equalize space between sections of Gallery

* feat: Resize Switcher modal popups to fit specimen cards cleanly

* feat: Placeholders for optional parameters

* fix: Make placeholders lighter

* ui: Pause visualizer while Switcher dialogues are open

* ui: Position Switcher popup centered over visualizer canvas

* fix: Restore Delft-submitted click behavior for closing Switcher

* ui: Invoke Switchers via current seq/vis looking and acting like entry fields

* fix: MageExchangeA icon should use current CSS color
* feat: human-readable URLs based on query strings

  To implement the planned URL search bar, it will become necessary
  to generate specimen encodings from sequence parameters without
  actually constructing the Sequence objects that have those parameters.
  That was not possible with the opaque base64 encoding.

  Hence, this PR switches to a human-readable encoding based on URL query
  strings as processed by the JavaScript standard URLSearchParams. It provides
  some functions to construct such encodings directly (rather than only
  mediated by existing Sequence and Visualizer objects). This PRI includes
  a backwards compatibility facility so that saved base64 encodings can still
  be interpreted. It also suppresses the Vue error trapping feature when
  in Workbench mode, which was and should continue to be very helpful when
  debugging.

* chore: remove unused hasStringFields function

* chore: update featured specimens to new URL scheme

* fix: parse query from path properly

* fix: Fewer url encodings -> more readable URLs

* fix: Suppress leading # in color query params

* feat: Lists separated by whitespace by default
* fix: All minor items from Delft final_fixes

* fix: Re-enable canned OEIS seqs with proper async computation

* doc: improve description of Sequence fill method

* chore: remove stray bold attribute; reword chaos comment

* fix: Right-docked tabs remain correctly positioned on zoom
…changes to Turtle and p5Template (numberscope#399)

In view of recent changes to the p5 visualizer class (parameter data structure and validation, handling aspect ratios and presketch function), this PR does the following:

    * updates the documentation making-a-visualizer.md to reflect these changes
    * updates the P5Template visualizer in the workbench to agree with said documentation and reflect these changes
    * cleans up the Turtle visualizer: make use of parameter-specific validation, reorder/prioritize the parameters according to importance, add a picture to documentation, change default values for aesthetics.

---------

Co-authored-by: Glen Whitney <[email protected]>
* refactor: move the OEIS search bar into its own component.

* doc: Basic Modfill -> two OEIS-based Specimens in Gallery

* feat: Dummy search results that pop up on input

* feat: Call a dummy search cmd on input

* feat: list of links and labels for search results

* feat: Clicking on a result adds it

* fix: When you click on link in results don't also add

* feat: Actually search the OEIS.

* feat: Cache search results and reopen search on click.

* ui: Layout and caption changes per design discussions

* fix: Never cache OEIS sequences with infinite indices
   This PR attempts to resolve numberscope#407. I added a half-dozen OEIS
   sequences as rapidly as I could and didn't get any errors.
   So hopefully this has squashed the problem, but it's hard to
   be certain.

* fix: Don't update URL until after parameter values have settled
   In order to be sure that parameter values have their "final" values
   in the case of an update (e.g., by calling the Paramable validate()
   method), it is necessary for a cascade of functions to be made async.
   This commit makes those changes and then awaits the validate() in the
   user interface, so that the URL won't be updated until all of the
   consequences of validate() have occurred (which may, for example,
   involve changing the parameter values, perhaps even the one that
   originally had its tentative value modified, necessitating the validate()
   call).

   Resolves numberscope#403.

* ui: show finger cursor over search results

* make oeis.org link open new tab

* info icon for info in search results

* ui: click off list of OEIS search results closes it

* fix: Restore sequences adding when you click on them in result list.

* fix: Various fixes to OEIS search bar/results per meeting

  - Return to the OEIS ID being a link but add the Wikipedia
    "external link" icon
  - Highlight the description of each sequence as you mouse over it,
    to provide clue you can click on it (to add as a sequence). The
    color used is a lightened version of the "primary" Numberscope color.
  - Display a message when there are no sequences to show as a result
    of a search. Note this can occur because there are _too many_ matches,
    or _no_ matches, and there is no way to tell which is the case from
    the OEIS api response, unfortunately. The message reflects this.
  - Fixed bug that results were being cached under the wrong search term
    if the searcher typed more while the search was executing.
  - Make sure that enter/clicking on magnifier always fires search

* fix: Don't make switcher cards for dummy message IDs

* fix: Really make sure magnifier puts/keeps search results up; reword message

* fix: catch errors in OEIS searching and report a 'soft' failure

---------

Co-authored-by: Kate Stange <[email protected]>
Adds a visualizer that displays the logs of sequence terms as a bar chart, with each bar a stack of sub-bars representing the prime factors of the term, with multiplicity. Mouseover and click can be used to highlight occurrences of a given prime.

Along the way, adds several enhancements to the visualizer infrastructure, including clarifying the circumstances under which presketch and setup are called; restricting some mouse and key events from reaching the visualizer to avoid interference with other widgets; allowing warnings as well as errors on parameter values; wait until initial sequence cache fill before validating visualizers; release focus from ui text entry fields when you hit enter;

Provides  docs for FactorFence, and updates general visualization authoring docs.

Adds a nice FactorFence example to the gallery

---------

Co-authored-by: Glen Whitney <[email protected]>
* test: Implement end-to-end testing with Playwright

  This is an in-progress cleaned version of the final PR (numberscope#361) of
  the Delft student user interface project, which it supersedes.

  Comments from the original PR that remain relevant:
  * The end-to-end tests run using Firefox and Chromium.
  * New tests are in the e2e folder.
  * The tests often depend on specific classes and IDs, so they may
    need to be updated upon changes to Numberscope.
  * The tests can be executed as the following npm script:
    `npm run test:e2e`
  * An interactive testing UI and debugger can be executed as the
    following npm script: `npm run test:e2e:ui`

  Caveats concerning trying this cleaned PR and its status:
  * Make certain to run `npm install` after pulling this PR.
  * Many of the tests do not yet pass, perhaps because of the "specific
    classes and IDs" point mentioned above and the fact that ui2 has
    diverged significantly from the Delft PR series.
  * Tests are not yet run automatically prior to commit.
  * I do not think there are any image tests yet, we need to try to add
    them.
  * Tests are not yet performed in the continuous integration checks
    to be run on GitHub; they should be.

* fix: Repair gallery/click on featured item test

* fix: Repair gallery/saving a specimen test

* fix: Repair scope tests.

* test: Run end-to-end tests prior to any commit

* test: Add image tests to e2e

  The strong desire for image tests led to a cascade of changes in this
  commit, mostly driven by the need to have reproducible images:

  - Removes all use of `sketch.noLoop()` and `sketch.loop()` in favor
    of the previously existing `stop()` and `continue()` visualizer methods,
    to allow:
  - Adds a `frames=NNN` query parameter to URLs to set the maximum number
    of frames a visualization may draw
  - Switches from the "static" instance of mathjs to a "dynamic" one, to
    allow its random number configuration to be controlled. In conjunction
    with this, moves all math functions into a single math module, as
    extensions of mathjs.
  - Removes all use of `Math.random()` in favor of the mathjs random
    generator
  - Adds a `randomSeed=AAAA` query parameter to URLs to make the mathjs
    random generator reproducible.
  - Documents all of the above changes.

* doc: describe running and creating code tests

  Also adds new tests for `src/shared/defineFeatured.ts` and corrects
  the documentation extraction facility for the package manager
  scripts.

  Resolves numberscope#25.
  Resolves numberscope#73.
  Resolves numberscope#246.

* test: Test that the caching mechanism won't double-calculate

   In other words, it should never call calculate twice for the
   same index. This is tested by 10K random accesses to indices
   less than 1M, followed by accessing the first 10K entries,
   followed by accessing the last 10K entries. Hopefully that
   should suffice.

   Resolves numberscope#54.

* fix: Prevent ModFill from hanging on extremely large input

  This is an initial pass at addressing numberscope#113.
  Note, however, that ModFill is not reporting to the person doing
  visualization that it is running with different parameter values
  than shown. So that still must be done, but for that part we will need
  a resolution to numberscope#112, which will be a sufficiently involve change that
  we should leave it to a spearate PR from this numberscope#420.

* doc: Update PR checklist.

  Resolves numberscope#174.

* maintenance: Remove and disallow trailing whitespace in code

  Resolves numberscope#219.

* maintenance: Make TypeScript target ES2022 in all cases.

  Resolves numberscope#226.

* maintenance: Run typecheck in CI

  Resolves numberscope#292.

* maintenance: Run GitHub CI on pushes to main as well

  Resolves numberscope#217

* test: Add OEIS Sequence-Visualizer test grid

  Use a list of "stressful sequences" to perform at least two image tests on
  each visualizer. As this change uncovered several existing errors, there
  are numerous other changes in this commit to return to a state of all tests
  passing. Here is an enumeration of other changes, in no particular order:

  * Change husky pre-commit actions so that the end-to-end tests are not
    run if there is an existing successful test since any files in the project
    last changed.
  * On most image tests, snapshot only the visualizer canvas.
  * Make 'axios' a runtime dependency, rather than just development, as it
    is used when obtaining OEIS sequence values in the running frontend.
  * So many other changes are made to Histogram.ts that this PR also adds
    hatching to show the elements with unknown factorization. Adds the
    "p5.brush" package to draw the hatching. Since this necessitates WebGL,
    adds a new 'P5GLVisualizer' base class for visualizers that wish to use
    a WebGL canvas.
  * Updates several dependencies to latest to make sure their out-of-dateness
    was not contributing to test issues.
  * Switches to the (ES) "module" import system from "commonjs" to ensure
    that was not contributing to test issues.

Resolves numberscope#294.

  * Specifies font locations in a way that vite understands how to
    relocate in both the 'dev' and 'build' versions of the app.
  * Properly marks both factor cache and value cache as empty at the
    initialization of an OEIS sequence.
  * Allows ±Infinity as valid values of INTEGER param fields for convenience.
  * Allows specimenQuery to take the output of parseSpecimenQuery to
    recreate the same specimen query as was parsed, for the sake of testing.
  * Turns off the browser default context menu on visualizers, which wasn't
    particularly useful and was interfering with the UI for some visualizers.
  * Fixes typos in Differences visualizer (this.first -> sequence.first)
  * Makes FactorHistogram visualizer a p5 WebGL-based visualizer, and adds
    hatching to the "0 factors" bar to indicate the terms with unknown
    factorizations.
  * Uses the WebGL default controls to implement panning, zooming, and (rather
    uselessly but somewhat spectacularly) rotating the plot in three
    dimensions for the FactorHistogram visualizer.
  * On FactorHistogram's first pass over the sequence data, collects the
    counts for each possible number of factors, as opposed to the number
    of factors for each entry, to avoid a possibly disastrously long loop
    on the second pass that accumulates bin counts. This also streamlines
    the computation of the largest number of factors in the data.
  * Factors out repeated code in FactorHistogram, for labeling bars and
    displaying text.
  * Fits the hover box in FactorHistogram to the text to be displayed.
  * FactorHistogram displays a temporary message if factoring is taking a
    long time.
  * Moves the bar labels of FactorHistogram just under their respective bars,
    to make sure they are visible.
  * Selects y-axis ticks at round numbers for FactorHistogram, and moves the
    tick labels closer to their ticks.
  * Prevents FactorHistogram from looping except when there is mouse activity.
  * Prevents ModFill from freezing up if too large a "mod dimension" is
    chosen. Note TODO: display a warning when a smaller mod dimension than
    requested is actually used.
  * Prevents Vue's reactivity system from attempting to modify the behavior
    of p5 sketches, using the Vue `markRaw` method. This change prevents
    some instances of infinite loops caused by cascading change notifications.
  * Tightens up the typing of P5Visualizer so that it is possible to derive
    another visualizer base class P5GLVisualizer from it. Also splits up the
    inhabit method so that P5GLVisualizer can modify it as needed.
  * Allow negative start indices for Show Factors visualizer.
  * Show at most 100 terms in Show Factors (no room on screen for more than
    that).
  * Updates TypeScript target versions of JavaScript

Resolves numberscope#226.

* refactor: better typing for param descriptions

* feat: Add ExtendedBigint ParamType

* refactor: Remove generic parameters from top levels of Paramable hierarchy

* feat: Uniform sequence bounds controls
  All params for controlling which terms of a sequence will be used in
  the visualization are removed from individual visualizers. Instead, there
  are uniform params in the Sequence classes themselves.

  Resolves numberscope#411.

  In this implementation, several other changes are necessary and/or
  expedient to make. In particular, the type of sequence indices is changed
  to bigint and bounds to ExtendedBigint (the union of bigint and ±Infinity),
  in preparation for addressing numberscope#455. Additional changes include:

  * Removal of SequenceDefault class, as everything now derives from Cached.
  * More care in caching of OEIS sequences, to provide some help with numberscope#459.
  * Individual-parameter validation functions now take a validation status
    to update based on the finding. This refactor avoids a frequent need to
    merge status objects.
  * Individual-parameter validation functions are called in a context
    of `this` set to the Paramable object, so that other data from the
    paramable can be accessed.
  * New `math` functions for dealing with ExtendedBigints.
  * NumberGlyph visualizer now tries to display as many terms as are
    available and will fit on the screen, except in case there are infinitely
    many terms available, in which case it still defaults to 64 terms. This
    change is needed so that the general length parameter for Sequences would
    affect what NumberGlyph displayed.
  * Puts reactivation of "known" OEIS sequences into a function
    rather than at top level of sequences.ts so that the timing of
    when it occurs can be controlled.
  * Renames property `_size` of P5Visualizer to `size` as multiple derived
    classes seem to need this value (i.e., not just used internally in the base
    class).
  * Makes sure the p5 loop is restarted if need be every time `.show()` is
    called on a visualizer.
  * Temporarily disables A000521 transversal tests until we can
    get incremental OEIS loading to work.
  * Adds a test for starting a visualization deep into a sequence.

* refactor: Apply OEIS modulus on the fly, so cache can point to communal one

* test: First working e2e tests with text and transversal

  Unfortunately, due to the intricacies of end-to-end testing with image
  comparison, this is a very large commit. It introduces running tests
  in a Docker container for the sake of reproducibility. (So note that
  henceforth you must have docker installed and running on your machine to
  perform testing, and hence to make a commit.)

  However, it turns out not all tests can be run in a Docker container --
  Firefox is not able to supply a WebGL context inside of Docker. Hence,
  some tests still need to be run directly on the host machine.

  To deal with all this, I felt it necessary to introduce the "make" tool.
  In particular, creating the necessary Docker image is slow, and I didn't
  want to have to repeat that except when truly necessary.

  That, in turn, means there are now many more configuration files, as
  well as auxiliary files created by the Makefile. The directory tree was
  becoming hopelessly cluttered. To keep things straight in my head, I felt
  it was really necessary to reorganize the directory structure. The biggest
  change is to put as many of the configuration files as possible in a new
  etc/ directory (the old-school unix name for where to put such things).
  I managed to get almost everything put in there. The one major holdout is
  the tsconfig files; I just couldn't find a way to get TypeScript to run
  without its config files at the top level of the project. Slightly
  annoying, but then, there are so many more annoying things about TypeScript.

  Anyhow, accomplishing the configuration file rearrangement resulted in
  updating eslint. Unfortunately, it went through a _major_ rewrite from
  version 8 to 9, and prettier-eslint has not been updated to work with
  the new version. I tried to work on that update myself, but prettier-eslint
  is incredibly intricate in the way that it handles configurations because
  it is trying to provide a great deal of flexibility and power, and it is
  trying to infer prettier configuration from eslint configuration (and maybe
  even vice-versa as well, I'm not sure). Since we were only using a tiny
  fraction of that power, it turned out to be significantly easier to just
  replace prettier-eslint with a custom script tools/prettiest.js that runs
  first prettier, then eslint. It also meant we could get rid of the
  lint-staged tool, so there is that. (While I was at it, I ran the "depcheck"
  tool and got rid of some other unneeded cruft that had accumulated.)

  But now it all works, and as a side benefit since just about every npm
  script now runs through make, you should never need to worry about
  running install before dev or build before preview, etc. Make keeps track
  of what depends on what and whether/how to redo the prerequisites before
  taking the requested action. Even better, if you just ran successful
  end-to-end tests, which take a while now, then `git commit` will not have
  to re-run them.

  Those are the major points. In addition, there are a number of minor changes
  resulting from errors uncovered in getting all the tests to work, etc:

  - Reordering of element attributes and other reformatting in vue components,
    because with the lint updates and reconfiguration, it now seems
    significantly stricter about vue templates.
  - First pass at an updated User Guide for ui2; it definitely needs more work.
  - Some other more minor documentation updates.
  - Updates tools/editor/autoformat.el so that emacs can use the new
    "prettiest" tool for formatting (even though I know I am likely the only
    one in the world who will ever use that).

* test: Add skipped tests reflecting known shortcomings

  As per Numberscope discussion today, this is the last missing element
  of the end-to-end testing PR. With these skipped tests, we are flagging
  that there are known concerns we want to resolve at least by beta, if not
  alpha. Also fixes a small gap in the docs for running from source that
  Aaron noticed in the meeting.

  With this, the PR will be marked ready for final review.

* fix: Generate docker image even with no existing test results

* chore: Attempt to run e2e tests in CI workflow

* test: No WebGL tests in CI :-(

* fix: format of Playwright reporter parameter

* chore: Try to grab GitHub CI actuals to make separate CI test file

* fix: oops test must succeed to generate artifact

* chore: Another try to grab ci snapshots

* test: Add in the extracted CI snapshots

* doc: Note that PR reviewers must run e2e tests themselves, too.

* chore: Stop grabbing snapshots when we don't need them

* fix: Changes as per review comments

* doc: more info on Docker (and fix remaining typo)

* test: Fuzz the pixel comparison in Firefox WebGL tests

* test: Really fuzz the pixel comparison in Firefox WebGL tests

* test: Check if docker tests passed before updating result directory

* doc: Improvements per code review

* doc: show the expected output of successful end-to-end test

* chore: remove stray comment that no longer applies

* fix: useful default camera controls for WebGL: left drag pans, wheel zooms

* fix: correct Husky action inclusion and test it

* fix: don't alter the URL just loaded, and reset frame limit on changes

* fix: adjust dragging, detected mouse position, and text size for zoom/pan

* fix: Keep the 'too many bins' message in a fixed absolute canvas position
…scope#463)

* doc: Harmonize guidance on git, cloning, and other workflows.

Resolves numberscope#221.

THere is still a signifcant amount of repetition/overlap among
CONTRIBUTING.md, and in the doc/ directory, onboarding.md,
running-from-source.md, and working-with-git-and-github.md. I wasn't
sure how we might compartmentalize these, because they are at
different levels of detail and for different audiences.

Since they are all now consistent with each other, I am not too fussed
by the repetitiveness. Maybe it's OK to give potential contributors multiple
places to find the information they need. But if the reviewer has suggestions
about how to reduce the redundancy, I am also open to making more changes.

* doc: Harmonize gitting-it-right with all of the other git/workflow docs

* doc: Changes pursuant to PR review
This fixes many issues with the Turtle Visualizer and implements several important new features. Here is a (probably not exhaustive) list:

* Adds "Fold rates" and "Stretch rates" parameters to allow animating the turtle instructions
* Allows all of the turtle instruction parameters to either be a single value applying to all elements of the domain or a separate value for each domain element. (Actually, it just repeats the last instruction element as many times as needed, which is occasionally useful in other ways as well.)
* Skips sequence entries not in the domain, rather than just stopping, but also checks that it is not skipping too large a percentage of entries
* Allows the drawing speed to be adjusted
* Allows the view to be panned, zoomed, and rolled
* Provides a few new (beautiful!) featured sequences
* Avoids restarting the path drawing on a resize
* Improves error checking of parameters
* Updates documentation
* Streamlines the parameter assignment scheme to minimize number of reinitializations at page load; this change improves the reliability of end-to-end tests

Resolves numberscope#76.
Resolves numberscope#223.
Resolves numberscope#412.
Resolves numberscope#453.
---------
Co-authored-by: Kate Stange <[email protected]>
Co-authored-by: Glen Whitney <[email protected]>
…ramable (numberscope#473)

* refactor: make overall status and status of each parameter part of Paramable

* doc: Describe updating CI snapshots, part 1

* doc: Describe updating CI snapshots, part 2

* fix: ModFill needs to recheck max modulus whenever modDimension changes

* fix: When size changes, presketch must be re-run

* fix: Changes as requested in review
  - Further detail on updating CI snapshots in docs
  - Catch empty formulas at validation time, not when they throw error later
  - Make descriptions and per-value validation messages each consistent:
    the former are full sentences with periods, the latter are phrases
    without periods
  - Add a FORMULA parameter type and use it for both places where formulas
    are used as parameters
  - Fix extraction of free variables in formulas
  - Rename the "modulo" parameter of ShiftCompare to "modulus"
  - Change the "log(EXPR^x)" in the default glyph growth formula in
    NumberGlyph visualizer to "log(EXPR)*x" to avoid crazy big exponentiations.
    Note this changes the visual results because the exponentiation was
    overflowing into floats large enough that the loss of accuracy made the
    following modulo operation a constant.

* chore: update GitHub CI snapshots for NumberGlyph

* doc: Fix typo and add message conventions
* fix: Make sure browser 'back' button actually updates specimen

  Prior to this change, the browser 'back' button would always update the
  URL, but the visualization would typically not change if you were going
  from one scope URL (`/?name=Blah&...`) to another.

  This PR remedies the situation by watching for route changes in the
  Scope view, and reloading the new visualizer and sequence **into** the
  specimen already being viewed. Implementing this tactic requires the
  ability to completely change the sequence and visualizer of a specimen,
  which previously did not exist.

  The change also enables delaying specimen initialization from Vue setup()
  time to just before the component is mounted. As a result, the Vue setup
  is not asynchronous, so the experimental Vue <Suspense> tag is no longer
  needed.

  Adds a test that the back button actually took effect, to the existing
  test that changes a parameter. (The extended test definitely failed in
  ui2 prior to this PR.)

  Uses the router to redirect from an empty URL to the last-viewed specimen,
  rather than the Scope component; this change avoids reloading that specimen
  when the URL is updated to reflect what specimen is being viewed.

  Finally, fixes a small previously-existing bug in which the Differences
  visualizer could never display more terms than were available in its last
  parameter settings, even if you changed the parameters to make more
  terms available.

  Resolves numberscope#414.

  Given the large number of internal changes to Specimen.ts, please
  in review do play with a veriety of visualizations to make sure all seems
  to be working well.

* fix: Make sure Thumbnails clean up their visualizers
* doc: Harmonize docsite style with Numberscope

* refactor: call checkParameters() when sequence changes

   Since checkParameters() is allowed to query the sequence, for
   consistency it must be called again when the sequence changes.

   Also updates the documentation on the "Building a Visualizer" page,
   adding an extensive lifecycle diagram that hopefully will clarify
   what happens when as a visualizer operates.

   Resolves numberscope#487.

* doc: Harmonize link styles with frontscope

* doc: Misc improvements
  * Admonition to be sure to clean up state in setup()
  * More thorough vetting of new versions in production in server admin page
  * Typos

  Resolves numberscope#319.

* doc: Update visualizer "Behind the scenes" documentation and move into source
   This commit updates the documentation of the VisualizerInterface,
   ParamableInterface, and Paramable base class, and relocates this
   documentation alongside the respective source code for ease of future
   maintenance. It also adds documentation for ParamType and ParamInterface.
   Resolves
   numberscope#425 (comment)

* doc: Split making-a-visualizer documentation as recommended

* doc: Convert diagrams to d2 and add more detail

* test: install d2 in GitHub workflows

* doc: Document INVALID_COLOR

* doc: User-guide documentation for the Sequence classes.
  These pages will be needed for numberscope#321.

* doc: fix unvisited link color in left sidebar

* doc: typo fixes etc per review

* doc: top-level page organization

* ui: Logo -> About, visualizer/sequence/documentation -> Help

* chore: update GitHub CI snapshots for new NavBar

* doc: Adjustments to NavBar per conversation

* doc: Hone Help menu further, crop sequence page images

* doc: Further revisions of lifecycle diagram as requested

* ui: Further refinements to controls, links, and popups as discussed

* doc: two tiny fixes per review

* doc: improve contributors table in about.md

* doc: add missing parens in lifecycle legend
* ui: Fix493: replace magnifying glass with help icon

* ui: Entire OEIS search result adds sequence; only link icon to OEIS

* ui: Scroll to and highlight new sequence cards, then let them fade

* feat: Add external links to OEIS for SpecimenCard and ParamEditor
numberscope#498)

* fix[Formula]: reject unknown function at parse to avoid throwing error

* feat: Convert formula calc errors to param tab warnings

* fix: Formula shouldn't throw when result is not a number.

* fix: Formula warning on conversion problem should show type.

* fix: Make sure warnings are shown every time Formula changes

* fix: Use the warnings themselves to know how many there are
* fix: Make sure warnings are shown every time Formula changes

* fix: Drag into sidebar always docks into an empty zone

  Prior to this PR, dragging a tab on top of a docked one would leave
  it floating over the docked tab. Moreover, if the empty zone in the
  sidebar was less than half the size of the tab being dragged, it would
  be impossible to dock the dragged tab at all.

  Now any drag into a sidebar with at least one empty zone results in
  a docking to the targeted zone if it is empty, or the first empty zone
  if not.

  Resolves numberscope#431.

  Note that this PR does not at the moment implement any resizing.
  There is resizing described in issue numberscope#431. However, now that this is
  working smoothly, it is making perfect visual sense to me that the
  dragged tab fits itself into the remaining empty space, leaving the
  already-docked tab alone, and in fact, this behavior seems to me
  preferable to what's described in numberscope#431. But if there remains a clear
  desire for the resizing as described in the issue, that can be
  implemented. Just let me know.

* chore: revert change to Formula.ts left over from Git manipulations

* fix: tab drags and docking/undocking preserve size whenever possible

   Also fixes bug that sometimes prevented docking in an apparently empty
   spot (the `empty` class was not being properly maintained).
@katestange
Copy link
Member Author

Ok, before moving on to documentation and snapshots, I think this needs a little feedback on how to handle the parameters. I think I have a tendency toward "poweruser" parameter setups, so I'm happy for some feedback how to handle things.

@katestange
Copy link
Member Author

Another thing I could use advice on is how to manage the lines drawn per frame. The reason shift-compare tended to hang is that it tried to draw the whole screen at once on difficult computations. I've made this one do some number of rows per frame so the picture comes in gradually. But I'm not sure how best to manage the number of lines per frame so that the sequence switcher in thumbnail mode (for example) is not laggy.

@katestange
Copy link
Member Author

There's another weird bug I'm encountering, maybe you can confirm/deny for me. Go to this url: http://localhost:5173/?name=IntegerStellar&viz=SelfSimilarity&width=400&distance=2&height=400&seq=Formula Do you see a nice picture, or a plain yellow screen? I sometimes have to change the Transformation Formula to m*1 from m (should be equivalent!) to get the nice picture to show up!

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

Hmm, divides was already in math.ts, so now it is doubly defined. Please remove the excess one, thanks. And the reason for b >= 0 && b <= 0 is that inequalities work with disparate numeric types (bigint mixed with number, for example), but equality testing throws an error.

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

Indeed, modulo and gcd will not be needed after the mathjs update, so no need for more modulo tests.

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

There's another weird bug I'm encountering, maybe you can confirm/deny for me. Go to this url: http://localhost:5173/?name=IntegerStellar&viz=SelfSimilarity&width=400&distance=2&height=400&seq=Formula Do you see a nice picture, or a plain yellow screen? I sometimes have to change the Transformation Formula to m*1 from m (should be equivalent!) to get the nice picture to show up!

I can confirm that from a cold load, the above URL is an almost-all-yellow screen, but that changing some of the parameters, e.g. just overtyping the default formula 'm' with an explicitly entered 'm', fixes it to the nice diagonal rows of different intensity dots.

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

A comment that is likely only of academic, not practical, interest: conceptually, this visualizer could be unified with Grid by (a) allowing a formula for the value to be entered in cell (x,y), e.g. gcd(a(x), a(y)) and (b) adding a color data type to mathjs and allowing color formulas based on the value c entered in a cell and the max entry M and min entry m, e.g, rgba(255, 255, 0, (c-m)/(M-m)).

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

I will take a shot at debugging the "weird bug"; I'll let you know if I succeed or stop working on it for now.

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

OK, I fixed the weird bug in the last commit (you didn't have the correct inputs in the default value for shiftFormula). But there is another bug: you are not handling cache misses correctly, in that you loop over 500 cells, and if you miss a cell because the try-catch in drawNew fails, you never go back to that cell. That leaves a blank gap in the diagram the first time you use a given sequence. It gets filled in if you change some other parameter than the sequence, because by then the cache is full. I will leave this for you to fix, but let me know if you need help.

@katestange
Copy link
Member Author

Hmm, divides was already in math.ts, so now it is doubly defined. Please remove the excess one, thanks. And the reason for b >= 0 && b <= 0 is that inequalities work with disparate numeric types (bigint mixed with number, for example), but equality testing throws an error.

Thanks, that was a manual merge error, fixed.

@katestange
Copy link
Member Author

I will leave this for you to fix, but let me know if you need help.

I believe I have fixed it. It simply retries the same location until the cache successfully returns.

@katestange
Copy link
Member Author

@gwhitney question: I'm trying to update snapshots. But this PR is failing not only on the new featured items, but also on a bunch of transversal items. Is it the case that the existence of a new visualizer has resulted in a different subset of transversal items, and therefore it's resurfacing out of date snapshots (for example Modfill)? That's what it looks like (e.g. some ModFill look out of date, some snapshots are just missing). If so, is it ok to go ahead with re-snapshotting those?

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 4, 2025

Yes and yes.

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 5, 2025

A comment that is likely only of academic, not practical, interest: conceptually, this visualizer could be unified with Grid by (a) allowing a formula for the value to be entered in cell (x,y), e.g. gcd(a(x), a(y)) and (b) adding a color data type to mathjs and allowing color formulas based on the value c entered in a cell and the max entry M and min entry m, e.g, rgba(255, 255, 0, (c-m)/(M-m)).

Actually, after diving a bit more into the review, I am now thinking this comment is of more than academic interest, and that we should actually think carefully about the design of Grid vs SelfSimilarity before proceeding with both/either. I came to this perspective after seeing that four of the five Gallery examples of SelfSimilarity are simply calculating an opacity based on a function of the (x,y) coordinates of a pixel, and two of them are just binary properties, and two of them don't even use the "distance" defined in the visualizer at all. So at least a few of them seem much more like "Grid" visualizations rather than "SelfSimilarity" -- the latter name anyway speaks conceptually much more of things like autocorrelation in a sequence, which doesn't really seem to be going on much in the examples so far.

So there is an underlying complicated visualizer here, that can do everything that either grid or self-similarity can. Call it maybe FormulaGrid: You get pixel coordinates x, y, and s (the spiral distance from the center), and function symbol a that computes the Sequence selected on an input, and then you can supply one (or more) formulas that compute an rgba color to draw in that cell. Perhaps there should be a shape (rectangle, small square, ellipse, small circle) -- associated with each formula if there is more than one.

Should we implement that visualizer? In its raw form, it will not be neophyte-friendly. But it could be very fruitful for "power users".

If we do implement that visualizer, what of Grid and SelfSimilarity? Do they become simply simplifying options/presets for FormulaGrid with a clever UI layout in an effort to make FormulaGrid more accessible at first? Or do they become separate visualizers in their own right, with their own specific controls focused on their particular perspectives on visualizing the sequence, perhaps with autogenerated links at the bottom of their parameter panes that take you to the FormulaGrid visualizer configured to reproduce that particular visualization?

I think I kind of like this latter option. In this scenario, we would have FormulaGrid as described above. And then maybe the current Grid would become the Spiral visualizer, focusing on settings/parameters that are interesting from the point of view of exploring number spirals, but the curious advancing user could transition to the FormulaGrid visualizer using the link provided. And analogously, the SelfSimilarity visualizer could focus on autocorrelation (broadly interpreted) of a sequence. And those examples like Modular Multiplication Table that really are best described just as a color function of (x,y) coordinates could be Gallery examples directly of the FormulaGrid visualizer -- simple cases of it that produce striking images, that would serve as an on-ramp to a rather high-level, abstract visualizer.

So @katestange @Vectornaut I'm going to hold off on further review until we have a chance to discuss these points and settle on a strategic or at least tactical direction here. Maybe we could even manage it in person at JMM... Or if you have a clear vision of how to proceed here, Kate, feel free to just share it.

@katestange
Copy link
Member Author

Hi @gwhitney @Vectornaut -- this is an important question, and it's been bouncing around my head a little too. I think Glen's proposal is an excellent one, and aligns with our goals of making a powerful but accessible tool. I don't see any reason to avoid duplication of functionality, when there's a different use-case for different access points to that functionality. Meaning, I don't think it's a problem that two visualizers can create the same picture, if they serve different communities to get to that picture. I think Glen's design -- having "on ramp" visualizers that are accessible and do "part" of the whole thing, is a lovely solution! Vectornaut, thoughts?

If we go this way, I'm assuming the code design is to implement the power visualizer first, and then have the others be derived from it or layered on top of it in some way (so they are shells that just call the power visualizer under the hood based on their parameters)?

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 5, 2025

Yes, I think the most practical implementation route would be to implement the putative FormulaGrid, and then see if we can derive visualizer classes from it with streamlined interfaces focusing on certain sorts of visualizations, but calling FormulaGrid under the hood. Would be ideal to have just one actual implementation of the drawing part -- that's the most DRY and maintainable.

@katestange
Copy link
Member Author

Ok, well, unless @Vectornaut objects, I think we should go ahead. What's the order of operations, do you think? Does it need to wait until bigint is in?

@gwhitney
Copy link
Collaborator

gwhitney commented Jan 5, 2025

It's certainly most convenient to wait until bigint is in. Along with/on top of that, we can add factorization, valuation, and elementary number theoretic functions like sum of divisors, number of divisors, euler's phi, etc. (some of those we can also submit to mathjs itself, but unlike the fixes to current bigint support we don't need to wait for those submissions to go in to proceed). We can add a color type to mathjs if we decide to go that way (I think we may as well). Then implementing FormulaGrid will be very straightforward and not a lot of code. The main design question on my mind for FormulaGrid is whether we want to allow the order in which the cells are filled in to be customizable: i.e., always left to right, top to bottom, or whether we want to allow them to be drawn along a spiral path for the spiral cases, etc?

It also occurs to me that by allowing hex and/or triangle shapes as well, we can lay out number triangles (like the arithmetic triangle) in their usual way to get those nice modular colorings with this same visualizer. We could allow hex/triangle spirals as well, I suppose.

@gwhitney
Copy link
Collaborator

NOTE that this PR is out of sync since the update to main to correct a contributor name. It will likely need to be rebuilt on top of the new ui2. But in fact, I think we are planning to rebuild it on top of a new unified underlying FormulaGrid visualizer, so that shouldn't really be any extra burden. Leaving this open as a placeholder until all of the details are worked out.

@gwhitney
Copy link
Collaborator

And in fact, based on #541, we are likely to just redo this after alpha, for beta, as (one of of two) skins on an underlying formula-based grid "pre-visualizer". So this PR will likely be closed at some point, but keeping it around for now as a placeholder for lots of ideas and prototypes.

@gwhitney gwhitney mentioned this pull request Feb 22, 2025
9 tasks
@gwhitney
Copy link
Collaborator

Should we close this, in the same way that I just closed #522 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants