Would it be possible to have the mod author(s) do some of the certification themselves as part of the approval process? Let them run the tools and provide the reports as part of the submission. Also, would it be beneficial to prioritize the incoming submission queue based on factors such as existing author reputation, availability of source, whether self certify reports are provided, etc?
Personally, I’m for either 1 or 2, but with a modification. Instead of relying on decompilation, force every build to be CI-provided and linked with a commit in a publicly available repository. 3 sounds good at first, but is really useless; having submitted a legitimate plugin is no proof that there isn’t some malicious intent.
Assuming the developer owns/has access to their own CI, what would stop them from replacing the JAR artifact with a malicious one so that users inadvertently downloaded malicious code?
Of course, that would mean Sponge standardizing on a single (or an approved list of) CI providers. For example, Travis shows a link to the applicable commit for each build, so the config could be easily inspected.
I’m leaning towards Option 1 or 3. Preferably review every file similar to Bukkitdev. Unless you guys don’t think you will be able to keep up with the submissions. I can imagine it’s hardly a fun job to do either.
Otherwise Option 3 would be viable as well as long as:
- All files still get reviewed in the end
- Unreviewed files are clearly marked
- Projects can be flagged by the community
What about spot checks?
Have the first file manually reviewed. And then randomly decide whether the next uploaded file should be reviewed.
Let the community also flag files, and review those files.
1 Like
Personally I am more for the 2nd option. I like being able to control my own downloads, be able to have download metrics because the files are being served directly by me.
I’m also not a fan of any system where it requires that you are forced to wait while someone else decompiles your code and tries to read it. In a situation like that you run into the same thing that BukkitDev had where people who didn’t know what they were doing were reading your code and flagging it inappropriately. Either that or you were waiting for long periods of time to get a file reviewed and approved that many times was a critical fix that needed to be out RIGHT THEN but because of overload on the resources (people) available it just wasn’t feasible.
5 Likes
Great idea! I feel like overall there should always be someone reviewing the files manually, because there are always ways to avoid the scanning. Basically it’s about being able to provide the plugin download as quick as possible and the hybrid system mentioned above would actually provide that. So it’s up to the server owner which state of the plugin (unreview/community-reviewed/fully-reviewed) is sufficient.
I honestly think more things should be automated and run by the community instead of a small handful of volunteers. With a proper system in place (some kind of public ratings/reviews) the community can easily determine the good plugins/mods form the malicious ones.
Also, since it is related, I think you should have a look at this thread (especially what I said in post#24): Plugin Annotation MinVersion and Target Fields - Reply #24. Having a system like this would be extremely beneficial to the community, imho.
The plugins should be hosted in a repository-style so plugins can easily rely on other plugins such as WorldEdit’s API or any other plugin’s API without the need for a separate repository.
The problem with a community run system without an approval process is that a file can easily be uploaded that completely violates guidelines and such. There needs to be some sort of approval process in place so the community members using the service can be protected.
1 Like
I really like the third method.
I think one issue that Bukkit had was the speed at which plugins were verified. Bukkit was limited by their technology and that they did not have nearly as many workers as would have been best.
I think that opening up the process to the entire community would be an amazing idea, as it would allow for people to assist with things to make everything go smoothly. Anyone who is so much of an a** that they find the need to put back doors in their plugins should not be allowed in the community, and I think that most people have that viewpoint.
I say have the third as it allows for a moderate security and high speeds.
Why not just keep using bukkitdev? 
I don’t want to keep helping Curse make money after what they did to me.
BukkitDev staff were all community members. We were not limited by our technology, we spent months developing our own tools to make the process go faster. We never sacrificed security for speed though. We were happy with approvals taking awhile if we weren’t making mistakes. If we got the approval time down to 30 minutes but let through backdoors then we had failed.
See the above response. BukkitDev staff were entirely community members (now paid Curse staff…). Nothing was stopping anyone from applying and we regularly reviewed applications. Putting the burden on everyone would not work. It seems like a novel idea but look at Spigot, they had plugins with backdoors forever that went unnoticed because they are “community reviewed” or something.
Hi. Thanks for calling me incompetent. I hope you can look at this response without a condescending attitude: if you really had to get bug fixes out so fast, 1: do QA before you mark something as release, 2: use a CI service. We only hired competent people to review files. We weren’t perfect, but we knew what we were doing. Files were safe.
This could be a decent compromise.
As long as you have access to the CI backend, you can upload a file and make it look like it came from a certain github commit.
@teozkr your idea simply isn’t feasible for reasons already stated.
Which is why we did completely proactive reviews by hand on BukkitDev. You can’t have speed and security unless you have hundreds of people that are willing, able, and can follow throw on slaving (yes, slaving) away at reviewing files.
2 Likes
- is the only safe system.
It can always happen that someone steals an account of a trusted dev!
There could be a prority based on the plugins popularity. So that important plugins will be checked immidiately.
Also review links to “inofficial downloads” (but allow them), and show an alert when someone wants follow such a link ("You are leaving Sponge Repo. We are not che
1 Like
I meant no offence in what I said. I just heard from a post that Curse made on the Bukkit forums that the back end technology used on the BukkitDev site was not the best, I wasn’t too sure about its credibility, I guess I was mistaken.
I really liked the almost guaranteed security full source scanning provided at Bukkit, but it seemed like more trouble than it’s worth to fully scan the jar every time.
I think a better solution would be to have people upload/commit their source to a private repository that only Sponge plugin reviewers could view. Then reviewers will only have to review differentials and trace as far back into the code as needed. If plugins haven’t been fully scanned in x amount of time, they can be flagged for full review on next update so that reviewer errors don’t persist.
CI can build versions from approved source.
2 Likes
It has already been done successfully by F-Droid. Yes, it would take some effort to get right, but it’s not impossible. Actually, here’s a workflow that ought to be able to work:
- Add a script to .travis.yml that prints out the SHA-256 of the artifact jar
- Add a deployment script to the .travis.yml that uploads to a Sponge-controlled server, also giving the contents of $TRAVIS_BUILD_ID
- Sponge server verifies that the uploaded file corresponds to the previously printed checksum, and that the .travis.yml contains a correct version of the script that should print out the checksum
I don’t like the idea of one good plugin and you get ability for automatic upload no approval. However, I think if they have maybe atleast 5 plugins and/or are a certain rank on the forum(this could be a plus) they automatically get it as they are trusted, but still I think every plugin no matter if they are made by the owner of this forums should run through a basic scan for exploits or backdoors because some people might even put them in accidentally(i.e if mysql is implemented and only if mysql is added to the filter) and they dont realize it, the system can pick it up.
Another thing is, how do you filter multiple languages? You couldn’t only do Java as some use MySQL and possibly other languages. So how do we approach this, find professionals in those fields to point out common exploits/backdoor methods? just curious.
I think requiring all projects to be open source(publicly or privately but visible to moderators) so that people can check the code more conveniently would be better and quicker since they can check code changes easily
3 Likes
That makes absolutely no difference. People could have a backdoor in the jar that’s not in the source. Reading decompiled code isn’t bad.
Plugins would be done in Java. Java has a MySQL library. If there are things like plugin loaders for different languages (BukkitDev had a few of those), we’d just have someone with knowledge of that language review that code in your example.
1 Like