Plan - Player Analytics, Sponge/Bukkit/Bungee/Velocity support

A new version has been released for Plan, it is available for download here.


5.4 build 1583 - Hotfix for MySQL issue

This release contains a hotfix for MySQL driver unloading issues on disable/reload, causing the plugin to spew a really long error message and failing to reload when using MySQL.
Driver is no longer unloaded during reload to avoid any issues.

This bug was introduced in dev build 1536 on January 6th, but it was not caught or reported.

A new version has been released for Plan, it is available for download here.


Change log 5.4 build 1628

Hello! This update brings some great database optimizations, beginning of Frontend BETA (Opt-in), as well as small fixes.
Read more below

Sponge API 8 is coming soon, but the database schema changes in this update need to come first to not leave servers unable to update to Java 11 stranded.

:warning: This update is backwards incompatible due to database schema changes.
Servers with old Plan versions should not be enabled after starting a server with this update installed.

How to prepare for this update

Networks: Make sure to update all servers that are connected to the database!
If an old version starts with the new database schema it will attempt to reverse some of the new changes.

Special thanks to Kopo, Pingger & KasperiP for contributions to this update.

Change log

Database optimizations (Schema changes)

A lot of work was put into analyzing bottlenecks in queries and slow query logs sent in by users.

Benchmarks using 17k players are promising, as /players page now loads in 5s vs 2+min before.

Database patches will apply on first enable of the update to modify the database schema. This can take up to 4 minutes. Networks: Only enable one Plan server and wait for it to patch database before starting other servers.

  • Changed database schema to optimize query speed and table size
  • If you’re using these columns via Query API you can replace your use case:
    • If you need uuid in ResultSet, use SELECT u.uuid ... JOIN plan_users u on u.id=<table>.user_id
    • If you have uuid or server uuid in WHERE, use server_id=(SELECT id FROM plan_servers WHERE uuid='<server uuid>')
  • Some queries that were identified as slow were optimized to perform better by changing the query
    • Geolocations queries
    • Ping country table query
    • Player table query
    • Other queries are also sped up by the uuid → user_id and server_uuid → server_id changes, as comparing integers is faster than Strings.
  • Extension data insert was made faster by keeping icon ids in memory for short duration
  • MySQL insert performance may be improved since auto-commit was turned off
  • Query page queries now use user ids instead of uuids - this fixes ‘Max SQL size exceeded’ issues.
  • Added a new index to plan_ping table to speed up ping queries.
  • Removed redundant uuid indexes that are now replaced with foreign key indexes for user_id and server_id, which reduces database size.

Frontend BETA (Opt-in)

Frontend is being rewritten in React one page at a time. This is to have more control over what is displayed on the website, since a lot of requested features will require more interaction or hiding features partially. - While doable with vanilla javascript, code would quickly become unmaintainable.

The goal is to get feature parity with the current frontend implementation by rewriting one page at a time and then switch over completely.

Some features will not use React frontend until everything is complete.

  • To Opt-in set Plugin.Frontend_BETA to true
  • Player page has been rewritten in React, other pages still use previous implementation.
  • Html Customization is currently incompatible with the beta and is disabled automatically.
  • Export is currently incompatible with the beta and old pages continue to be exported even if beta is enabled.
  • Automated browser testing was improved a lot and all pages are now tested for basic javascript errors.

Data gathering

  • Fixed AFK time not being gathered on Fabric, the movement detector was incorrectly firing every time a player position packet arrived (Happens once per second).

Config

  • Default Player head image address was changed from cravatar to crafatar. Remove the whole row from the config and reload Plan if you wish to use the new setting.

Locale

Kopo did a massive locale system improvement in order to support translations on the frontend side, implementing yml conversion, namespacing all of the existing locale keys and writing json endpoints for frontend to load locale - as well as the code responsible for loading those using i18next.

  • Locale files now use .yml format. Any locale files you may be using are converted automatically.
  • The new React Frontend allows changing language on the frontend on a per-user basis with a dropdown menu - This is currently only available on the rewritten pages (player page)
    image
  • KasperiP fixed a typo in Finnish locale

Export

  • Players page is now exported more frequently
  • Exported pages should no longer show the Logout button

Extensions

  • Essentials Economy was split from Essentials Extension. Now there are 2 extensions managed separately. This was done since the economy calls were triggering updates too frequently. If you have disabled Essentials Extension you may need to disable EssentialsEco Extension separately
  • Fixed NPE related to LuckPerms Extension
  • Fixed SQL syntax error related to LuckPerms Extension
  • Fixed issues with KingdomsX Extension when using newer version of KingdomsX (They changed names of some of their packages)
  • Pingger added support for 1.18 stone blocks to LogBlock Extension

QoL changes

  • /plan servers command now lists Plan version on each server so it is easier to keep track of servers being out of date.

Bugfixes

  • Fixed negative register dates in player dat files breaking the database
  • Fixed ConcurrentModificationException related to AFK tracker
  • Prevented use of incorrectly formatted WorldAlias regex settings
  • Fixed Plan failing to enable if database query took too long during enable.

A new version has been released for Plan, it is available for download here.


Change log 5.4 build 1633

This update brings a couple important bugfixes to build 1628. Please read change log of 1628 if you’re running older build than 1628.

Change log

Database

  • Fixed databases failing to patch schema if plan_users or plan_servers were missing rows that had uuids in other tables.
    This affected roughly ~1 in 80 servers based on reports and metrics.
    This problem could occur if server had shutdown at a specific time during player join, or if someone had deleted rows manually from plan_users or plan_servers.
    • The fix deletes dangling data so that the rest of the patches can execute properly.
    • If you have run into this problem on your server this update should fix your database automatically.
  • Fixed SQL errors in ViaVersion Extension and ProtocolSupport Extension

A new version has been released for Plan, it is available for download here.


Change log 5.4 build 1690

This update fixes some annoying bugs and prepares the plugin for the move to 5.5 and Java 11 while keeping this version compatible for those unable to update their servers to Java 11.

If no further critical bugs surface next update will be 5.5.

Change log

Plugin stability

  • Fixed issue where Plan would prevent server enable from continuing if database was not responsive during enable.
  • Fixed 2 crashes when database write latency is high
    • Fixed thread starvation crash during high latency when using ViaVersion or ProtocolSupport
    • Reduced chance of out of memory crash when during high latency by starting to drop plugin data transactions for a while if queue grows over 500 transactions. (The transactions continue if queue returns to <50)

Database

Join address schema change

  • Join addresses are now stored in plan_join_address table and id referencing that is added to plan_sessions table for time based analysis implementation later.
    • Backwards compatible schema change - old versions will insert ‘unknown’ as each sessions address.
    • plan_user_info.join_address column was not removed, but its use is discouraged for 3rd parties as its use may be removed in the future.

Fixes to patches

  • Fixed UserInfoOptimizationPatch resetting join_address data when applied
  • Fixed typo in RemoveDanglingServerDataPatch that caused it to fail
  • Possibly improved speed of RemoveDanglingUserDataPatch

Other fixes to database

  • Prevented transactions from executing if database ran into fatal error. This reduces amount of errors logged on a failed patch
  • Fixed error related to user_id constraint violation for Ping and Session store transactions. If they execute faster than the user register transaction the player will not have up-to-date information until the proper transaction executes, but the storage of other data will no longer fail.
  • Prevented ActiveCookieStore related DbOpException if plugin failed to enable

Website

  • Server page > Performance now lists Average players under Performance as Numbers.

Frontend BETA

  • Implemented player ping graph on the player page when opted into frontend beta
  • Fixed nickname list not showing colors properly
  • Fixed terminal icon not loading when user registered with console
  • Fallback to using relative address if Alternative_IP is misconfigured to reduce confusion. Previously nothing would load if address was different, now there’s a chance something loads.

Placeholders

  • Wrote tests for placeholders
    • Fixed player_recent_kill_{1-10} and player_recent_death_{1-10} off-by-one mistake and related error for n=10.
    • Fixed player_unique_players_today erroring if there had been no sessions during today.
  • New placeholders
    • %plan_player_current_session_length%: The length of active session (Online time)
    • %plan_player_current_session_length_raw%: unformatted ms
    • %plan_player_latest_session_length%: The length of active session or previous session if not online
    • %plan_player_latest_session_length_raw%: unformatted ms
    • %plan_player_previous_session_length%: The length of previous non-active session even if online
    • %plan_player_previous_session_length_raw%: unformatted ms
  • player_ placeholders now support adding :playername or :uuid at the end of placeholder to show data for another player
    • Note that if the player by given identifier is not found the placeholder is not replaced.

Other bugfixes

  • Console will now log colors properly on Bukkit and Bungee based servers
  • Fixed some _raw placeholders from being formatted
  • Fixed Error related to GriefDefender Extension

A new version has been released for Plan, it is available for download here.


Change log 5.4 build 1722

This update fixes fabric 1.19 support and join address gathering.

Special thanks to Kopo for contributions to this update.

Change log

Fix join address gathering (No longer ‘Unknown’)

An oversight in the way the player leave listener code was structured caused the player join address to be removed from memory before its linking to the session that was ending.

It is now fixed.

Fabric 1.19 support

Kopo fixed fabric pipeline for 1.19 - Unfortunately the changes are incompatible with older versions of fabric. If you need older version of fabric use older version of Plan.

Frontend BETA

  • /players page rewritten in React now available when opted in to frontend beta
    • Note that no fixes to functionality were added to it yet, just implementation in React.
  • Server page rewrite is making good progress and is about 75% complete.

Server identification

  • Possibly fixed ServerInfoFile.yml clearing and server duplicating if MySQL is busy during enable

A new version has been released for Plan, it is available for download here.


Change log 5.5 build 1951

This release contains update to Java 11, Sponge 8, Fabric 1.19.1, as well as Webserver rewrite to Jetty. In addition it contains new features and improvements, and more progress on Frontent BETA in the form of rewritten server & network pages.

Special thanks to Vankka, Kopo, Drex & stashenko for contributions to this update

Change Log

Breaking changes

This version updates config so backup your config in case you need to revert back to 5.4!

  • Java 11: Plan now requires Java 11 or newer to run. Java 8 is no longer supported.
  • Sponge API 8: Sponge API 7 is no longer supported.
  • Fabric 1.19.1: Fabric 1.18 is no longer supported

If you can’t update, you can continue using 5.4 build 1722 even if connected to same database as newer versions. (No database schema changes!)

Supported APIs

  • Updated to Java 11, which was required by Jetty and Sponge API 8.
  • Vankka implemented Sponge API 8 changes.
  • Drex implemented support for Fabric 1.19.1

New features

Jetty Webserver

New webserver implementation speeds up response transmission times considerably

  • Replaced Sun Webserver with Jetty (Eclipse Jetty | The Eclipse Foundation), an open source embedded webserver library.
  • Plan now supports HTTP/2
  • Plan now supports TLSv1.3 for HTTPS, and browsers have a larger selection of cipher suites they can use.
    • Users of PlanCert.jks (Included in the jar) are encouraged to create their own self signed cert. I had to implement some extra code to support expired certificate (Since PlanCert.jks inside the jar has expired), and that may not be as secure as creating a new self-signed .p12 - Instructions will be added to the wiki, but the plugin also logs the instructions if you use that cert.
  • Fixed gzip compression for responses (speeds up transmission)

Access log

Access logs are now stored in Plan database in plan_access_log table.

  • You can also log access to console with Webserver.Security.Access_log.Print_to_console (default false) in the config. This is useful for debugging reverse-proxy issues.
  • The IP of each request is stored in the database - as it is necessary for security (a valid legal basis for use with GDPR).
  • The request logs in database are removed after Webserver.Security.Access_log.Remove_logs_after_days (default 30)

Swagger

Plan json endpoints are now documented with Swagger available on the webserver itself. You can access the Swagger documentation from /docs on your Plan webserver after updating.

Data Gathering

  • Fabric should now gather join addresses correctly. Previously player IPs were gathered by mistake. To clear the invalid data use /plan db removejoinaddresses {servername} once you install the plugin.
  • Added ‘Data_gathering.Preserve_join_address_case’ setting (default false) that attempts to preserve join address case. Note that some servers/clients may lowercase addresses regardless.
  • Possibly fixed an issue where Quit event is processed before Join, leading to long incorrect sessions being stored on next join/shutdown
  • Possibly fixed an issue where saving sessions that ended on shutdown failed when another join had just stored one of the join addresses that were previously unseen. (It would retry next reload and probably succeed - so low priority issue.)

Frontend BETA progress

Enable with Plugin.Frontend_BETA setting in the config. Note that Html Customization and Export are not yet supported in the beta.

  • Rewrote /login page in React
  • Rewrote /server page in React
    • You can now switch between visualizations for ‘current playerbase’ and ‘latest join address’ pie-graphs
      visualizer-gif
    • Join addresses by day graph is now available to analyze incoming join addresses further.
  • Rewrote /network page in React
    • You can now switch between Map projections in Geolocations
      projections-gif
    • You can now sort servers in server list by specific values
      server-sort-gif
    • Join addresses by day graph also available for network page
    • Added Players online to network performance tab - data from multiple servers can be viewed in one graph on network performance tab
  • Fixed various issues and cleaned up overall appearance

You can track progress of frontend beta here

Bugfixes

  • Fixed lag caused by large tab completion results if server had a lot of players and someone tab completed /plan player .
  • Fixed empty cookies preventing Plan login due to error
  • Reduced maximum join address length to 191 characters due to limitation of old MySQL versions

Configuration

  • Added plan.command permission (default: true) on Spigot servers to allow hiding /plan command from tab completion by negating this permission
  • Config structure was modified to be valid yaml so that other programs can validate it. The config is updated automatically with current values of old config versions.
  • Increased default inactive player removal threshold 180 days → 3650 days (10 years). This change is not applied automatically to old configs.
  • stashenko updated RU locale
  • Added /plan db migrate_to_online_uuids command to help migrating from an offline-mode server to online-mode
    • Moves data of offline uuids to online uuids (Looks up online uuid from Mojang)
    • Combines data of offline uuids and online uuids if both exist
    • See /plan db for more help

Extensions

  • Plugin method calls should no longer be made during Plan disable.
  • Updated PlotSquared extension, should now register correctly
  • Updated Floodgate extension, should now register correctly
  • Updated Nucleus extension for Sponge 8
  • Removed RedProtect support for Sponge (No longer supported in Sponge 8)

Automation

  • Dev builds for every commit is now available DEV build pipeline (Click on title and scroll down to Artifacts, note that you need to log in to Github.)

A new version has been released for Plan, it is available for download here.


5.5 build 1953 - Hotfixes to build 1951

  • Fixed extensions with only PLAYER_LEAVE not getting updated on player leave
  • Fixed data truncation error related to join addresses on restart when trying to store sessions from last shutdown
  • Self fix: If you’re unable to login after updating (usually with reverse-proxy setups), clear cookies.

Change log of previous release, build 1951

A new version has been released for Plan, it is available for download here.


5.5 build 1969 - Bugfixes

Special thanks to PikaMug for contributions to this update!

Change log

Website

  • Fixed login and register pages saying ‘You need to specify a password!’ incorrectly when using non-default Locale
  • Frontend BETA: Fixed players table on network page linking to wrong URL

Velocity

  • ProxioDev fork of RedisBungee is now supported for Velocity to gather correct network online player count.

Extensions

  • PikaMug updated Quests extension, adding new datapoints and fixing non-repeatable default quests showing as times completed: 0.
  • Fixed Extension tab icons showing wrong icon (Showed the icon for first datapoint)

MySQL

  • Fixed SQL exceptions related to ONLY_FULL_GROUP_BY mode being enabled.

A new version has been released for Plan, it is available for download here.


5.5 build 1984

This update contains small improvements.

Special thanks to DubHacker for contributions to this update.

Change log

  • [Webserver] Fixed login redirecting back to /login if multiple cookies were present (for example CloudFlare cookies)
  • [Locale] DubHacker updated DE locale

A new version has been released for Plan, it is available for download here.


5.5 build 2100 - 6 years of Plan!

This release contains a lot of things. Frontend BETA enters next phase since all pages have been rewritten in React, next steps are export and html customization implementation for it. This update also contains data gathering fixes, additions, new placeholders along with other bugfixes.

Today also marks 6th year of Plan development! As is customary at this point there’s a Year in Review -post after the change log.

Special thanks to Vankka, angela0930 for contributions to this update

Change log

Data Gathering

  • Removed data after null-byte in Join Address data. Bungeecord and Velocity use the handshake-packet to forward UUID and IP information of players, and since the join address comes from handshake packet, that data ended up in the join addresses Plan gathered. Forge Mod Loader forwards its version information in the same handshake packet in similar fashion. Discarding data after the null-byte should resolve any issues.

Frontend BETA

  • Names of new players are now shown in Light Green in PvP tables when they are the player who was killed within 24h of their first join. This is to help find players who kill newer players.
  • /query page has been rewritten in React.
  • /register page has been rewritten in React
  • Fixed plugin cards overlapping when moving to different page on a table
  • Fixed /network/… redirecting to /network
  • Fixed colors of buttons when hovering mouse on them

I’m excited, the frontend part of rewrite is complete and I can move to implementing export and html customization for the new frontend.

Export

  • Experimental version of export for React version is now enabled if Frontend BETA and Export are both enabled. This will place index.html at the root of the export directory along with all the react files. I have not yet been able to test it properly, so it might be completely unusable. Just throwing it out there.

DataExtension API: Component API by Vankka

If your plugin’s data contains chat colors or mini-message information, that information can now be included so that it will eventually show up on the website. Check documentation for the new API here

Extensions

  • Vankka fixed error that occurred when deleting LuckPerms groups
  • Vankka fixed LuckPerms not showing colors properly on the website

Placeholders

Added new placeholders

  • %plan_time_active_{day/week/month}%
  • %plan_time_active_{day/week/month}_raw%
  • %plan_top_player_kills_{day/week/month/total}_{n}%

Locale

  • angela0930 updated ZH_TW locale

Other

  • Fixed ConcurrentModificationException related to world aliases
  • Fixed some queries erroring in Strict group by mode for MySQL
  • Fixed Ore not showing Plan sponge version
  • Fixed removal of cookies that expired while server was offline
  • Fixed RemoveUsernameFromAccessLogPatch applying every time
  • Changed “Database can’t keep up with transactions” to less dire message
  • Fixed SQLITE_CONSTRAINT_NOTNULL when request uri is null

Year in review: Year 6

This year has been a lot different from the previous five, since there has been a lot more things going on in my personal life. Last year I got a job, and in January I started writing my Master’s thesis. It was very tough balancing all that on top of developing Plan on the side. I had to take some months off development entirely with a hiatus in the summer and fall to finish up the thesis. In September I returned the thesis for review and I’m happy to now have graduated! Things definitely calmed down after finishing up the thesis and resuming Plan development after the hiatus.

The running theme this year has been steady and stable changes. Inspired by the answers to last year’s poll, the goals for this year were to improve the loading speed of the website, as well as visualizing more of the gathered data.

There’s been a lot of work done this year towards those goals. The database schema was changed drastically, the webserver was changed to Jetty, and Frontend BETA is rewriting the code running in browser to React. Alone each change brings small improvements, but together they improve things a lot.
I’m very excited for the future, since the React frontend speeds up development of new things considerably thanks to environment where changes can be seen instantly. Previously I would have had to compile Plan and restart the test server – It takes 1 second vs 3 minutes it used to take to see changes now.

Main goals for 2023 is to get the new frontend out of Beta after feature parity with previous frontend is reached (Export and Html Customization still to go), and after that it’ll be time to tackle a lot of tickets with ’Visualization’ tag. A lot of the ongoing work has been done to make the future changes easier to do, which should speed up feature additions in the upcoming year.

I’m very happy that all the automation created in the past years is now finally paying off, helping keep confidence in that everything works as intended when changing something. I love that it’s now possible to deliver development builds fast, even within the same hour when someone is asking for help.

Special thanks to Github sponsors, donators, discord moderators, support, and contributors this year! Extra special thanks to Kopo & Vankka for their constant input into the project.

This year in numbers

  • From build 1516 to build 2100: 584 commits released as updates
  • From #43 to #20 in top plugins by rating – and keeping the #8 spot in top spigot plugins by rating.
  • Installations has stayed stable at over 4000 servers
  • From 59k lines of code to 64k (not counting extensions or web code)
  • From 51.3% test coverage to 57.4%
  • 194 issues and 32 (+247 dependabot) pull requests closed this year

Have a good one! :slight_smile:

A new version has been released for Plan, it is available for download here.


5.5 build 2121

Another release to add 1.19.3 support to Fabric and release all the goodies developed since previous update :slight_smile:

Special thanks to DrexHD for contributions to this update.

Change Log

Data gathering

Fabric

  • DrexHD updated fabric code to be compatible with 1.19.3 changes

Frontend BETA

  • Fixed login and register redirects to wrong place
  • Fixed Playerbase overview 30 days and Now being wrong way around on the table
  • React Export is now functional (At least on webservers that serve the bundle at /, further testing needed if your export is on /stats/ for example.)
    • If Frontend beta and export are enabled, the old html files are no longer exported.
    • Extra index.html files are exported to redirect back to the React bundle to allow sharing the URLs to others. This is because in the React frontend the address is handled by React Router and clicking most links on the pages makes no extra requests for html. Reloading on the other hand needs html file, so the extra files redirect to {address}/?redirect={address you used} - so that you end up where you want.
    • Exported pages have a different icon next to last refresh timestamp to visually distinguish from screenshots which one is being used.
    • Known issue with export: The json files are constantly being reloaded by the browser if they’re old

A new version has been released for Plan, it is available for download here.


5.5 build 2144

This build contains various bugfixes.

Change log

Join Address Data

  • Any join address data that has null characters (gathering was fixed in previous update) is now cleaned on first startup - the data after null character is cleared, and the valid data kept. This should help with join address tab not loading in many cases. This was thoroughly unit tested to ensure it works properly.

Query page

  • Fixed Query results erroring with 500 due to mistake in the code not placing boolean parameters in the query parameters.

Frontend BETA

  • Fixed timezone not being applied to graphs (All line graphs)
  • Fixed exported pages constantly updating data in the background

A new version has been released for Plan, it is available for download here.


5.5 build 2150 - Hotfix

This update contains a hotfix to build 2144. New installations after build 2100 are not affected - The bug affected instances that were updated from versions prior to build 2100 on networks or fabric servers. Sorry for any inconvenience it has caused. More about the bug below.

Fixed bug in BadJoinAddressCorrectionPatch

An unfortunate typo in session to join address id correction code caused all sessions to get invalid join address id. Instead of correcting invalid ids to correct ids, it changed correct ids to incorrect ids. Any installations where the broken patch ran lost their join address data.

Symptoms of the bad patch:

  • Playtime data too low or missing on player pages
  • Activity index differs between player page and player list
  • Join address data shows no data

Fixes in this update:

  • Fixed the typo, now the patch works as intended and corrects join address ids.
  • Added a second patch that attempts to recover at least some of the missing data by using latest join address in plan_user_info table for installations that ran the bad patch. This is a best-effort solution since the original data was deleted by the bad patch, so some granularity like player changing the address they have used in the past was lost.
  • Playtime and activity index values should recover since the issue was caused by join address ids pointing to invalid numbers.

A new version has been released for Plan, it is available for download here.


5.5 build 2163

This update contains performance improvements and subdirectory support for the new frontend.

Change log

Database

  • Fixed concurrency bottleneck where write and read operations interfered with each other, limiting to one query or transaction from executing at the same time. This bottleneck occurred since the access-lock designed to prevent database operations during schema modifications was still enabled after the schema modifications already completed.
  • Disabled BadAFKThresholdValuePatch - This patch was written to fix bad data input from version 4.5.2 which is no longer being used according to metrics, so this patch can be disabled. It was sometimes executed if a player joined a server and never moved.

Frontend BETA

Export features are now complete, up next is Html Customization.

  • Implemented support and tests for reverse-proxy setups with subdirectory proxy_pass settings (Eg. address.com/plan/)
  • Implemented support and tests for Export to a subdirectory (eg. /var/public_html/plan/ accessed from address.com/plan/)

Webserver

  • Added a read-write lock to json_cache so that files are not read while being written. This might solve some randomly occurring issues.

Plugin Enable

  • Incorrectly written lines in unsaved-sessions.csv during plugin disable are now ignored - a warning is printed instead of an exception stacktrace.

A new version has been released for Plan, it is available for download here.


5.5 build 2172 - CRITICAL security vulnerability fix

This build contains a fix to a CRITICAL SQL Injection vulnerability, as well as fixes to minor security vulnerabilities.

Yesterday (2023-01-14): Finding a minor Path Traversal security vulnerability lead to a throughout process of labeling all untrusted data in the codebase, and during that process a critical SQL Injection vulnerability was also discovered. When exploited successfully SQL Injection allows a malicious actor to read any data from the database and change or delete data. This may expose user salted+hashed Plan web user passwords or other data in the database.

It is recommended to update as soon as possible, even though exploits for the vulnerability may not yet exist in the wild.

The fix has been backported to build 1722 Release 5.4 build 1722.1 ¡ plan-player-analytics/Plan ¡ GitHub for those that need it.

This is a first time a this high priority vulnerability affects Plan, so I’m a bit overwhelmed, but I’m hoping to address this vulnerability professionally by releasing a fix in a timely manner, and keeping exact details undisclosed for now to give users time to update.

Change Log

Fixed CRITICAL SQL Injection vulnerability

Details
Vulnerable versions: 5.2 build 1168 to 5.5 build 2163

if login is enabled: Malicious users with permission level 1 (plan.player.other) or 0 (plan.server) can access an endpoint which was found to contain an SQL Injection vulnerability.
if login is not enabled: Any malicious actor can access an endpoint which was found to contain an SQL Injection vulnerability.

Mitigation if you are unable to update

  1. Enable https and login so that less users have access to the vulnerable endpoint.
    https://github.com/plan-player-analytics/Plan/wiki/SSL-Certificate-(HTTPS)-Set-Up
  2. Enable IP Whitelist so that less users have access to the vulnerable endpoint.
Webserver:
  Security:
    IP_whitelist:
      Enabled: true
  1. if unable to update or secure the server, disable Plan Webserver. This option is good if you want to delay updating to a more convenient time.
Webserver:
  Disable_webserver: true

Other fixed security vulnerabilities

  • [Minor] Fixed Path Traversal vulnerability where attacker could gain read access to .css, .js, .png, .woff, .woff2, .eot, .tff files anywhere on the host machine if Customized_files.Enable_web_dev_mode setting was set as true
  • [Minor] Fixed XSS (Cross site scripting) vulnerability in Whitelist deny 403 -page when attacker routes traffic to Plan through a reverse-proxy with malicious X-Forwarded-For header
  • Removed untrusted data from exception messages used within the plugin
    • [Minor] Prevented potential XSS vulnerabilities in Not Found page when untrusted data could enter the error message
    • [Minor] Prevented potential XSS vulnerabilities in Internal Server Error page when untrusted data could enter the error message
  • [Minor] Prevented malicious Hello-packet from breaking Session serialization to CSV on server disable if join address had a ; character in it

Locale

  • Updated Finnish (FI) Locale

A new version has been released for Plan, it is available for download here.


5.5 build 2272

This update enables the new React based frontend by default, speeds up loading various parts of the website, and fixes a couple of issues.

If you are using Html Customization, it is now possible to migrate to the new system, see Html Customization migration guide to React ¡ plan-player-analytics/Plan Wiki ¡ GitHub - You can use Plugin.Use_Legacy_Frontend setting until your migrations are complete.

Back up your config in case you need to revert to previous version. Updating should be easy, simply replace the jar.

Change log

React Frontend (Previously called Frontend BETA)

The frontend rewrite is complete, so the new frontend is now enabled by default. Plugin.Use_Legacy_Frontend (default: false) setting still allows using old frontend for a while until it’s completely removed.

Here is a summary of improvements this brings if you have not participated in the Beta:

  • Faster loading time since less data requests are made at once
  • Improved mobile navigation
  • New features
    • Page navigation button for switching between pages
    • Switching language on the frontend
    • Join address tab
    • Visualizer switches for some graphs
    • Average players online data to Performance tabs
    • Interactive ‘?’ help for Activity Index and New Player Retention (These were the two most common questions on how they work)
    • Redesigned Network > Servers tab
  • Easier to maintain and develop further

Changes from previous update:

  • Added a page navigation button that allows moving to different servers and other pages easily. This replaces the ‘Back to main page’ button. You can switch between the same page for two servers (eg. Move from Server 1 > Performance to Server 2 > Performance in one click)
    image
  • Improved mobile navigation. With the navigation button this should help mobile users a lot.
  • Interactive ‘?’ help for Activity Index and New Player Retention (These were the two most common questions on how they work)
  • Fixed network server list saying “No servers installed” while servers were being loaded.
  • Fixed page translation issues
  • React was updated to version 18
  • Javascript APIs for extending the page programmatically were implemented APIv5 PageExtension API ¡ plan-player-analytics/Plan Wiki ¡ GitHub
  • Fixed join address data breaking the page when visualized as a table
  • Fixed issues of plugin cards overlapping when switching between plugins of two servers on player page

New feature: public_html

A new feature in the webserver allows hosting any web files on the Plan webserver. Please note that any files placed in the public_html folder (/plugins/Plan/public_html by default) can be read by anyone who knows the address to the webserver even if you have login enabled. The folder can be configured with Webserver.Public_html_directory setting.

The main purpose of this feature is to allow Html Customization of the React bundle Html Customization ¡ plan-player-analytics/Plan Wiki ¡ GitHub

Webserver

  • Implemented HTTP Caching: Browser will now cache some responses and avoid sending unnecessary data if it was already loaded. This can improve page loading times from multiple seconds to milliseconds.
  • Implemented public_html feature that allows hosting custom files from a configurable folder Webserver.Public_html_directory (default /plugins/Plan/public_html). This can be used to host http-challenge file for certbot and other files.

Database

  • PlayerTableRowPatch should no longer be re-applied all the time
  • Optimized server player table query: /server/players now loads much faster. Tested optimization: 4s → 500ms: 8x improvement
  • Optimized server latest join addresses query: /sever/join_addresses now loads much faster. Tested optimization: 19s → 150ms: 120x improvement
  • Optimized /v1/network/servers endpoint, got a 66% speed increase, so Servers tab on network page should load faster.

PlaceholderAPI

  • Unregister placeholder extension when Plan disables: this possibly fixes an issue where PlaceholderAPI would log errors when Plan disabled before PlaceholderAPI.

A new version has been released for Plan, it is available for download here.


5.5 build 2307

This release has some new tools for analyzing player retention, as well as further improvements to the website.

Special thanks to TheLittle_Yang for contributing to this update.

Change log

Player Retention Analysis Graph

This update adds a graph to network and server pages (Playerbase > Player Retention) that has options to:

  • Draw graphs at different time resolutions
  • Limit input data by time
  • Group players by register date or join address
  • Visualize player retention in different ways
    • Time since registration date
    • Playtime
    • Date
    • Cumulative player gain
    • Percentage / Player count / Stacked player count

Any and all combinations are allowed which allows extensive analysis of player retention. Help sections attempt to make the data understandable and show examples.

Data gathering

  • Attempt to correct register dates if they report a date in the year 1970 (Epoch millisecond 0 = Jan 1 1970).

Website

  • Any extension tables can now be sorted
  • Fixed Network > Servers Overview Quick view graph not loading
  • Fixed sidebar being transparent in nightmode on mobile
  • Fixed Activity index help modal not drawing the index function after certain x threshold
  • Fixed data not updating when switching from Server to Network page in cases where they shared same components (eg. Playerbase graphs, Join address graphs, etc)
  • Improved player overview card layout on mobile
  • Reduced font size on mobile when in portrait mode, so that more tables remain readable

Extensions

  • Fixed NPE when player joined and FloodGate had not loaded its API

Locale

  • TheLittle_Yang updated Traditional Chinese (CN) Locale

A new version has been released for Plan, it is available for download here.


5.5 build 2391

This is a feature packed update, as it brings support for multiple proxy servers, IP allowlist CIDR and Dynamic DNS support, Fabric 1.19.4, and lots and lots of improvements.

Special thanks to Kopo, WolverStones & inductor for contributions to this update!

Change log

Fabric 1.19.4

  • Kopo updated Fabric support to 1.19.4, older versions are no longer be supported by this version
  • Kopo fixed Gamemode change event not triggering
  • Fixed a dog killing something crashing the server when running older fabric version

Multi proxy support

Plan can now be installed on multiple proxy servers. This is useful for networks that divide their players between multiple proxy servers.

  • You can now name proxy servers with Server.ServerName config option (to distinguish in Performance tab)
  • Server.IP setting is no longer required to be set, since some proxy servers may have their webserver disabled. It’s recommended to have only a single Plan webserver running.
  • If multiple proxy servers have their webserver enabled, the commands that have links link to one of them.
  • If all webservers are disabled, but export is enabled, the commands that have links link to one with export enabled.
  • Network online graph stacks the player online counts of all the proxies if RedisBungee is not used. Non-stacked version can be checked from Performance tab.

Known limitations: Online sessions/status are not synced on multiple proxies

IP Allowlist improvements

  • IP Allowlist now supports a lot more notations for easier use: CIDR, Wildcard, IPv6 with omitted zeros. See comment above the setting for examples.
  • IP Allowlist now supports dynamic DNS addresses. Add “dns:{address}” to the allowlist to have Plan resolve the IP address of the dynamic DNS. This may be useful for those without static IP addresses at home.

Database

  • MySQL databases using incorrect character set are now corrected to use utf8mb4 collate utf8mb4_general_ci upon startup. This should prevent some errors from popping up.
  • Fixed join address by day graph error if MySQL strict group by policy is enabled.
  • If Plan detects that two servers are storing data with the same UUID, a warning will be logged to console once every 30 minutes.
    • This usually happens if ServerInfoFile.yml is copied between servers during installation.
    • /plan info command now shows the ServerUUID of the server so that it is easier to debug which server it is
    • The detection looks if previous tps was stored < 30s ago, which suggests two servers are storing data as the same one
  • Add Database.MySQL.Max_Lifetime setting. This setting can be used if database connection keeps timing out

Website

  • Added option to disable registering new users Webserver.Security.Disable_registration (default false)
  • Improvements to network performance tab
    • Added some information why TPS, entities or chunks is not available if only proxy servers are selected
    • Reduced resolution of the data to minutes, so that all servers with data show up in the tooltip when hovering over the graph
    • Fixed server selector not allowing to change servers in some cases
    • Fixed the graph re-rendering when selecting different servers even though the data wasn’t reloaded
    • Added warning if none of the selected servers have sent data over 30 days
  • Improvements to network server list
    • An icon is shown if server hasn’t sent data for some time (Red triangle if no data in last 24h, Grey triangle if no data in last 7d, Grey archive icon if no data in last 30d)
  • /plan server command now gives link to /server/UUID instead of /server/Name
  • Fixed configured theme not applying by default
  • Extension data with colors or links now render properly
  • Fixed the logged in player’s head image not loading on the top right

Locale

  • Custom locale files (locale.yml) are now updated with new translation lines whenever server starts
  • WolverStones updated Czech (CS) locale
  • inductor updated Japanese (JA) locale

Extensions

  • Fixed ExceptionInInitializerError in Extension construction causing Plan to not enable properly
  • Fixed exception related to LibertyBans Extension
  • Fixed exception related to Towny Extension

A new version has been released for Plan, it is available for download here.


Change log 5.5 build 2461

This update contains bugfixes. This is probably the last version that will support ‘Plugin.Legacy_Frontend’ setting. Version 5.6 will remove the old frontend code so migrate your html customizations to React as soon as possible.

Special thanks to DrexHD, lis2a & ringoXD for contributions to this update.

Change log

Data gathering

  • AFK time is now updated when server shuts down to avoid AFK players from appearing active if they remain AFK during shutdown.

Fabric

  • DrexHD updated to support fabric 1.20
  • Added support for netty based socket addresses to possibly fix geolocation gathering issue.
  • Fixed plugin incompatibility with Geyser and other plugins with default mixin reference map (Thanks to onebeastcris for reporting this)

Website

  • Fixed permission level 2 redirect to /player/{uuid}
  • Fixed issues with Query page not allowing to change Registered between or Played between filter dates
  • Latest join address pie is now sorted by ‘Most players first’

Database

  • Added experimental support for MariaDB 11. (MariaDB 11.0.2 has a data insertion bug, use 11.1.1 if possible). This is automatically enabled if MySQL driver fails to connect to the database.
  • Fixed issue with plugin groups query related to Vault on Query page
  • Fixed join address truncation error when join address was over 191 characters.

Locale

  • lis2a and ringoXD updated Japanese (JA) Locale

A new version has been released for Plan, it is available for download here.


Change log 5.6 build 2614

Hello! This version contains a hefty set of new features, bugfixes and usability improvements. Highlighted features include Web User Access-Control, Plugin Version History and ‘Click to see Who’-features. There are a lot more new things so check out the change log.

Updating to this version modifies database schema
There is a guide for updating to this version here: Migration guide from Version 5.5 to 5.6 ¡ plan-player-analytics/Plan Wiki ¡ GitHub

Special thanks to yu_solt, ToxiWoxi, Kopo, xlanyleeet, Jumala9163, ringoXD, Dreeam-qwq & jhqwqmc for their contributions to this update!

If you would like to support the project financially, please consider sponsoring the main developer.

Change Log

Web user access control (HTTPS required)

A more granular user access control has been requested for quite a long time. Now you can limit users to only see smaller parts of the website. This feature is documented in Web permissions ¡ plan-player-analytics/Plan Wiki ¡ GitHub though you will find some help embedded on the /manage page.

Migrating from old version with permission_level adds groups ‘legacy_level_0’ etc with permissions that match previous behavior, users are linked to them automatically.

In order to edit web permissions on the website you need to set at least one webuser as admin with /plan setgroup {username} admin.

In order to keep /plan register working, give your players ‘plan.webgroup.{group_name}’-permission in your permission plugin (such as LuckPerms) for the group you want them to have. If player has none of these permissions they can’t register at all.

Plugin Version History (HTTPS required)

Plan now gathers plugin versions on server start. Any modifications are timestamped. This can be useful when tracking performance impact of plugin updates. This data was possible to gather from all server platforms.

Seeing this data on the website requires HTTPS to be set up because some server admins may consider this data sensitive. Without HTTPS the data endpoint is disabled and the data is only visible in the database. After HTTPS is set up admins can control who sees this data through web permissions.

On network page it is possible to check plugin versions of any server on the network. This can help you keep up to date with your plugins.

Click to see Who

click-to-see-demo

You can now click & drag on Server Calendar (and the new Network Calendar) to see who was playing on specific days the calendar data is about. On Geolocations Map you can click on a Country to see who has joined from that country.

These features utilize the existing Query features so it was relatively simple to implement. If you have more ideas where you would like to see who data is about, you can make a suggestions on Github

Website

New features

  • Player tables now show Average, Best and Worst Ping for all players
  • You can now choose visible columns on any table (such as Players tables and plugin tables.)
  • Network Calendar was added to Network Overview, similar to Server Calendar
  • Navigation button can now go to Plan Error Logs and Swagger Docs
  • The bigger features mentioned above

Improvements

  • Network Performance now disables TPS, Entities and Chunks buttons if there’s only Proxy servers in the selected dataset.
  • Player page Plugins navigation now only shows servers that the player has data from.
  • Fixed some table text color issues, especially in Night Mode
  • Fixed issues with reverse-proxy https falling back to relative addresses in frontend when proxy-mode https was not in use. (Plan would assume address is http:// but browser would have https://, leading to different start for the address, which was interpret as incorrect address.)

Removal of old frontend files

With the React rewrite being complete an in use by most users, this version deletes the old frontend files from the jar. This reduces jar size.

If you still haven’t migrated your html customizations this version does not load them anymore. Migration guide: Html Customization migration guide to React · plan-player-analytics/Plan Wiki · GitHub

  • Removed Plugin.Use_Legacy_Frontend setting.
  • Removed any code related to this setting, old behavior and the old web files.
  • Moved most html rendering from backend to the frontend.
  • DataTables did not support rendering React inside table headers so the library was removed and all functionality used by Plan rewritten in React. The visual differences should be minimal.

PageExtension API changes

  • You can now use web permissions with WebUser
  • Registering custom web permissions is possible either by overriding Resolver#usedWebPermissions, or through ResolverService#registerPermissions. There are two methods in ResolverService, one which grants existing groups the new permissions based on existing permission.

Webserver

  • Address of any webserver that is enabled can now be used by commands (Useful for users with 2 game servers without proxy)
  • Disabled X-Forwarded-For reverse-proxy warning temporarily since it was giving false positive warnings often.
  • Fixed out of date information appearing on the /player page due to HTTP caching

Endpoint changes

  • Added /v1/preferences and /v1/storePreferences endpoints that has some default formatting values coming from config.yml and user specific preferences if they have them set. Preferences UI is not yet implemented so for now this only returns the defaults.
  • Deprecated /v1/players endpoint, replaced with /v1/playersTable which gives data in raw format instead of formatted.
  • Added Group Management related endpoints /v1/webGroups, /v1/groupPermissions, /v1/permissions, /v1/saveGroupPermissions, /v1/deleteGroup (Only enabled with https)
  • Added Plugin history endpoint /v1/pluginHistory (Only enabled with https)

Database

  • Fixed join address truncation error on backup
  • Added tables plan_web_group, plan_web_group_to_permission, plan_web_permission, plan_web_user_preferences & plan_plugin_versions
  • plan_security column permission_level was removed and a foreign key column group_id added
  • Inactive player cleaning is now disabled on Game servers if Proxy server is in the database to avoid confusing situations where limit configured on Proxy server doesn’t apply.

Spigot

  • Possibly fixed Ping not being gathered on Spigot 1.20+ servers
  • Dreeam-qwq fixed an error that occurred when Join Address didn’t contain a port number.

Fabric

  • Fabric 1.20.2 support was implemented by Kopo. Older versions of fabric are no longer supported.
  • Plan now uses semantic version for fabric metadata

Sponge

  • ToxiWoxi fixed SpongeForge failing to load Plan due to unspecified LuckPerms dependency version in Plan

Locale

Some functionality was added to make life of translators easier.

  • Added setting which lists untranslated keys Plugin.Logging.Log_untranslated_locale_keys (default false). This lists anything that has default value so it may give some false positives.
  • Custom locale.yml file is now reloaded when it is modified.
  • Ukranian Locale (UK) added by xlanyleeet
  • Japanese Locale (JA) updated by yu_solt and Jumala9163
  • Simplified Chinese Locale (CN) updated by jhqwqmc
  • Translation of some data was moved to frontend so that it changes when you change the language.

Placeholders

  • Fixed off by one mistake in %plan_top_…% placeholders where 1 would give the 2nd highest and 10 nothing.
  • Added %plan_player_geolocation% placeholder

Extensions

  • Fixed BuyCraft not sorting dates correctly
  • Fixed an issue where rapid fire Vulcan AntiCheat violations would cause database exceptions.
  • Updated Lands API to newer version
  • Possibly fixed an issue where Litebans data was not being updated
  • Fixed issue where ViaVersion would show no data on network page