Release Notes

Version 1.0.8 (2022-05-29)

Quick Count Feature

You can now quickly see the total number of rows returned by your query, in the right-hand side of the toolbar area. This would previously have required a =COUNT style formula.

If there are multiple grouping levels (one-to-many relationships), the row count is done at whichever grouping level the cursor is currently located.

By default, only the retrieved number of rows is shown (e.g. ">99 rows"). You can toggle the indicator to get full counts. In this mode, Ultorg will run separate COUNT queries in the background, when necessary, as you move the cursor or modify your perspective.

There's a short demo of this feature here.

Screenshot of the quick count indicator in the toolbar

General Improvements

  • Use native file choosers on Windows, e.g. for the Connect to File action.
  • Fix numbers showing up as e.g. "99\u00A0999" on locales that use space as thousands grouping character (#21). In general, avoid showing the NBSP character as an escape code.
  • Support proxy server auto-configuration (PAC, used in certain corporate environments).

Data Sources

  • Text Files (CSV/TSV):
    • Parse up to 2048 columns, so long as at most 250 are visible as once in a single perspective query. Show informative error messages when limits are exceeded. (#23)
    • Improve character set auto-detection in the presence of Cyrillic letters.
    • Allow all-midnight timestamp columns to be parsed as a simple date column.
  • MySQL: Fix "Client restrict authentication plugin" error when connecting (#7).
  • MariaDB: Add a workaround for a MariaDB bug which could cause incorrect formula results.
  • PostgreSQL: Fix "cannot use serializable mode in a hot standby" error (#13).
  • SQLite:
    • The JDBC driver for SQLite is now bundled with Ultorg.
    • Fix a NullPointerException when working with columns of an undeclared type (#27).
    • Make the TEXT function work properly, including on columns of an undeclared type.
  • Oracle:
    • Retrieve the initial list of database tables in a single efficient query. The operation should now be quick even when there are thousands of tables.
    • Properly handle VIEWs with no primary key and where ROWID is not permitted.
  • Microsoft Access:
    • Fix a bug in the date/time type detection logic.
    • Avoid opening "linked tables" on paths outside the main database file.

Version 1.0.7 (2022-04-23)

Revamped Text File Connector

Support for connections to delimited text files (comma-separated, tab-separated etc.), has now been completely revamped:

  • New UI to configure and preview data import from delimited text files.
  • New configuration options to set character encoding, delimiter characters, quoting style, headers, decimal character (comma vs. period), date format etc.
  • Date and number parsing now supports a broader range of formats.
  • All configuration options have auto-detected defaults. This includes a custom-developed character set detector, tailored for the historical idiosyncracies of CSV/TSV formats.
  • An option to "show only rows/columns containing special/non-ASCII characters". This is useful when trying to guess or verify a file's original character encoding.

Screenshot of the Connect to Text File dialog

Visual Query System

  • Proper support for a UUID data type (on PostgreSQL only for now).
  • Add the fromEpoch formula function (create timestamp from UNIX epoch time).
  • Have the "Refresh Table Metadata" action automatically disconnect and reconnect any underlying JDBC connection. This may be necessary after external changes to the database schema and/or access privileges.

Layout System

  • If text values contain invisible control characters, indicate them with greyed-out escape codes.
  • Fix a text alignment bug that could happen when printing in landscape mode.

General

  • Update libraries and bundled JDBC drivers, and add a vulnerability scan to the release process.
  • Various other improvements and bug fixes.

Version 1.0.6 (2022-02-17)

Data Sources

  • Remove an unintentional 4KB size limit on cell values in CSV/TSV files.
  • For Presto connections, recognize either the Trino or the Athena JDBC driver.
  • Avoid creation of multiple extracts for the same table in certain cases.

Application UI

  • Various cosmetic improvements in the user interface (borders, alignment etc.).
  • Auto-size the Fields/Filter popup, for better positioning near screen edges.

Data Editing

  • Allow inserted records to be shown in relations joined on a non-primary key.
  • Keep edits visible even when fields within a multi-field key are reordered.
  • After applying edits in a perspective over multiple data sources, automatically refresh underlying extracts so that the changes become visible.
  • Disallow data editing in extracts, and improve related error messages.

Visual Query System

  • Make the function documentation popup visible when starting to edit a formula.
  • Some bug fixes relating to the Collapsed Join feature.
  • Some bug fixes relating to columns that are no longer available.

Layout System

  • Avoid very wide columns that could occur before fixed-width crosstabs.
  • Adjust heuristics for text breaking in labels, and for vertical labels.
  • Improve vertical positioning of text within cells.
  • Display non-latin text properly on MacOS as well (completes work from 1.0.5).
  • Fix uneven letter spacing (kerning) when printing on MacOS.
  • Have Ctrl+Home/Command+Up move the cursor to the current table column's label.
  • Handle the Backspace keystroke.

Version 1.0.5 (2021-11-19)

General User Interface

  • New, polished look for all parts of the user interface outside and around the visual query interface: buttons, popup menus, sidebars, windows etc. The same theme is now used for Windows, MacOS, and Linux, with some platform-specific customizations to to blend in with each native platform.
    • The new look should appear automatically, but can also be explicitly selected by opening the Internals→Platform Options dialog and selecting Appearance→Look and Feel→"FlatLaf Light".
      (Might be necessary if experimental settings were changed in the past.)
    • On Windows, the main window title bar and menu bar have been collapsed, leaving more space for showing actual data.
    • Lots of smaller adjustments to user interface colors, margins, borders etc.
  • Properly display text in non-latin scripts such as Chinese, Japanese, and Arabic, if the relevant base fonts are installed on the operating system. This is useful e.g. when viewing database tables containing a variety of Unicode strings. (Works on Windows and Linux, but not yet in the perspective area on MacOS.)

Visual Query System

  • Fix a bug in an interaction between the collapsed join feature and the Custom Group action.

Version 1.0.4 (2021-10-31)

Visual Query System

  • Initial support for range filters, i.e. "greater than", "less than", and "between" conditions in the Filter popup. Previously, such conditions required the use of a formula. Range filter conditions are suggested automatically if the user enters one or two values in the Filter popup's "Search" field. This applies to fields with data types such as Number or Date.
  • Fix errors that could occur when using the Filter popup, and avoid some non-intuitive behaviors that appeared in the previous release.
  • Have Clear Filter behave consistently with Filter on multiple selection.
  • Improve the behavior of the Fields popup's "Show Group Hierarchy Only" option when working with schemas that have cycles in their foreign key relationships.
  • Avoid showing certain primary key fields as collapsed join fields.
  • In the formula bar, avoid showing unnecessary precision for numbers and times.
  • Generate SQL queries that may run faster in certain cases, in particular where IS NULL/IS NOT NULL constraints or Custom Group levels are involved.

External Data Sources

  • Avoid triggering connection attempts from background tasks. The username/password dialog now pops up only after an explicit query action.
  • Avoid flashing a perspective warning while loading new table metadata.
  • Gracefully retry database queries for certain connection timeout errors.
  • Fix a bug where Refresh Extract failed due to metadata not yet being loaded.
  • Fix authentication failures for some MySQL connections (+possibly others).
  • Fix bugs which could occur if a new primary key appears for an external table.

Automatic Layout Generator

  • Fix a bug where the time portion of certain timestamps were hidden by default.
  • Fix incorrect column width measurements for certain collapsed join fields.
  • Various smaller adjustments to auto-generated layouts.

Version 1.0.3 (2021-08-16)

Platform-Specific Improvements

MacOS

  • Add native support for the new "Apple Silicon" (ARM/M1) based computers (select MacBook Air and MacBook Pro models), which were launched in November 2020. This makes queries run faster and the user interface noticably snappier.
  • Avoid a spurious "Allow Notifications?" prompt on MacOS.

Linux

  • Automatically fix font rendering problems which occurred on Linux when using the KDE desktop environment.
  • Better-looking tab components and borders in the user interface.

Windows

  • Improvements in the general look of the user interface, including a slightly larger text size.

Visual Query System

  • Further improvements to the Filter popup, to allow the user to configure filters without waiting for the complete filter options query to complete. Any value typed into the "Search" box will become immediately available.
  • Added the diffMonths, diffYears, and addYears functions to the formula language.
  • When Create Extract is invoked on multiple selected perspectives, show an option to create either multiple extracts or a single extract of UNIONed tables.
  • Suppress connection dialogs if the user has already clicked Cancel once. (More work on improved connection management is expected in the future.)
  • Make Copy (Ctrl+C) on partial query results more informative.
  • Fix some minor corner cases (relating to missing tables, and collapsed joins).
  • Tuned the bundled PostgreSQL database to improve extract loading performance.

Version 1.0.2 (2021-07-18)

Extracts & Data Sources

  • The Refresh button (Ctrl/Command+R or F5) will now automatically refresh any extracts used to evaluate the current query. For file-based data sources (Excel, CSV, or Microsoft Access files), this will only be done if the file has actually changed since the last time the extract was refreshed.

    For now, schema changes (added/removed tables or columns) must still be manually refreshed via the "Refresh Table Metadata" action.

  • The Stop button will now also stop any extract loading operations triggered by the current query.
  • Improve the handling of table columns which no longer exist in the external data source, or where the column's data type has changed. (More work will be done in this area in the future.)
  • Avoid spurious connection attempts (+username/password prompts etc.) for data sources not selected or related to the current perspective.
  • Adjust rules for data type detection for CSV/Excel/Google Sheets sources.
  • Experimental support for Amazon Athena (Presto) connections. Remote queries and most of Ultorg's standardized formula functions are supported.

    To connect to Amazon Athena, first download the Athena JDBC driver from https://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html and add it in Ultorg via Internals→Services→Databases→Drivers→New Driver.

  • Bulk export: Large database tables, or extracted query results, can now be efficiently exported to CSV files.

    To do this, select one or more database tables or extract tables in the "Folders" sidebar, right-click the selection, and select "Export to CSV". Hold down Ctrl/Command to select multiple items. To export query results, first dump them to an extract using the "Create Extract" action.

    As in previous releases, it is also possible to export to CSV by selecting one or more columns in a query result (Ctrl/Command+A to Select All) and invoking Copy (Ctrl+C), e.g. for pasting into Excel. This will copy only the portion of the query result currently loaded in the perspective tab.

  • The Create Extract action now adds missing column labels automatically.

Layout System

  • Adjust borders in generated table layouts, for a slightly simplified appearance.
  • Improve auto-setting of table column widths for cases with many empty cells, and when a significant amount of extra space is available.
  • Allow Print to PDF on MacOS when no physical printers are installed.
  • Increase the maximum number of crosstab columns that can be shown in a layout.

Visual Query UI

  • When opening the Filter toolbox, show a partial list of options immediately, in case the exact filter options query takes some time to execute. Avoid interrupting ongoing perspective queries unnecessarily.
  • Add some missing keyboard shortcuts for spreadsheet-like cursor movement on MacOS (Command+Arrow, Ctrl+Home/End, Alt+Arrow/Space, Ctrl+Space) and Windows/Linux (Alt+PgUp/PgDown).
  • In the Custom Join dialog, automatically expand relevant data source folders.
  • Simplify the Fields/Filter/Format popup.
  • Added the INT (parse integer) formula function, on each supported backend.
  • Further improve options displayed by the Custom Group action in some cases.
  • Make the Unhide action more intuitive when invoked in a Custom Group.
  • Fix query UI bugs that could occur after canceled connection attempts.

Application UI

  • Improve the appearance of UI tabs and borders on Windows and MacOS. Fix most remaining scaling issues on HiDPI/Retina screens.
  • Improve clicking and drag/drop behavior in the Folders and Fields sidebars.
  • Reset sidebar/toolbar state on startup. Fix bugs in "Show Editor Toolbar".

Version 1.0.1 (2021-04-01)

Startup and Installation

  • On MacOS, Ultorg is now packaged into a proper DMG file, with the customary "Drag icon to Applications folder" window. The application has been signed and notarized with Apple, as is required on recent MacOS versions.
  • Fix a problem that caused the application to fail to start on Windows if the containing folder contained certain non-English characters.

Visual Query System

Simplifications

  • Get rid of the Collapse Duplicate Rows feature, which used to be shown in the main context menu. It was seldom useful to change the defaults for this setting, and the option to do so frequently confused users. Equivalent queries can always be constructed using the more flexible Custom Group feature.

    The grouping level options shown by Custom Group have been adjusted slightly, to make certain query changes more straightforward to perform.

  • Get rid of the "At Most One Child Per Parent" indication. It was of limited usefulness, took up space, and confused some users.
  • Get rid of the "In Primary Key" context menu item. It is easier to see the primary key in the field selector.
  • For Custom Group levels, show the "Unhide" label badge at the group level only, as opposed to on every primitive field. This declutters the layout.

Bugfixes

  • Properly handle the case where a previously available table disappears from an external data source (one specific bug fixed here).
  • Fix a rare assertion failure in the Delete Custom Group action.
  • Fix a rare assertion failure when deleting multiple-selected fields.
  • Fix tooltips cutting off the last word when showing warnings or formula parameters.

Performance

  • Fix certain slow queries involving aggregate functions inside multiple custom grouping levels.
  • Make the SINGLE formula function more efficient.

Layout System

  • Add online documentation for property editors in the Format tab.
  • Add an experimental "Strip HTML Tags" formatting option. (Not yet exposed in the Format tab.)
  • In Data Editing Mode, show rows pending deletion with a strikethrough style.

Data Sources

  • Upgrade the MariaDB/MySQL JDBC driver to the latest version. This fixes problems with primary key detection on MySQL, and adds support for the sha256_password/caching_sha2_password authentication schemes (which are now the default on new MySQL installations).
  • Upgrade the Microsoft SQL Server and PostgreSQL JDBC drivers to their latest versions.
  • Attempt to fix the authentication page for Google Sheets on certain Linux systems; one user reported the browser window not opening.
  • Allow metadata operations for different data sources to happen concurrently. This prevents one non-responsive data source from blocking all others.

Earlier Releases (feature highlights)

Release Category Feature
0.9.8 General Fix startup problems on certain systems
0.9.7 Visual Queries Adjust retrieval of crosstab columns and filter options
0.9.6 General Persistence of all application data across restarts
Data Sources Connect to CSV files and Google Sheets workbooks
0.9.5 General Folders for data sources, tables, and perspectives
Data Sources Extracts and extract-based UNIONs
Visual Queries Perspectives over multiple data sources
Visual Queries Blue arrows to indicate instantiated tables
Visual Queries Custom sort orders, and automatic weekday/month sort
0.9.4 Visual Queries Improvements in formula editor and field selector
0.9.3 Data Editing Data editing improvements, including Undo/Redo
Data Sources Lazy loading of external table metadata
Layouts UI for formatting customizations
Visual Queries "Contains" filters
Visual Queries Adjust and simplify actions in context menu
0.9.2 Layouts Formatting shortcuts in toolbar
Visual Queries Make the Fields tree more usable
0.9.1 Layouts Print & Print Preview support
0.9.0 General Retina/HiDPI screen support
Layouts Collapsed joins
Layouts Robust form and crosstab layouts
Layouts Automatic center alignment for certain data
Visual Queries Custom Group and Move to Root actions
Visual Queries Copy-to-Clipboard
Visual Queries Fix most critical bugs and performance issues
Visual Queries Improve sorting behavior
Visual Queries Enforce distinction between Tables and Perspectives
0.8.2 Data Editing Basic data editing
Data Sources Bundled PostgreSQL database
Data Sources Bundled JDBC Drivers
Layouts Frozen value headings in form layouts
Visual Queries Complete set of documented formula functions
Visual Queries Fully robust query evaluator