Maybe a mixture of method #1 and #3, having an eye on the newer devs and plugins projects which are less than 3 month or so online. But if the plugins are around for like half a year, switching to an automated scan would be good. Also, there could be some kind of recommendation to include the source files (e.g. check the “include source” option in eclipse export process).
Maybe there could be a 3-stage system like this (working with colors ):
First, there is the stage where the moderators approve the first version of the plugin to verify that there is no backdoor. The plugin gets a yellow “moderator approved”-sign in the plugin-list.
Next it gets a green “scanner approved”-sign in the list
If the source is public/included, and more than X community members have reviewed the project, stage 2 is entered: A blue “community approved”-sign in the list
Maybe the order of the stages is another, but I like the idea of such a system. It would be the most secure and would guarantee that older/known plugins are in the state of being very secure to use, while newer plugins will make their way to that state after some time.
If you’re going to work with some sort of scan system you could put a delay on the scan. That way people wont be able to test various methods of circumventing that scan in a small time frame by trial and error.
I like 3 the best. The cons you mentioned are well worth the expedited review process - I don’t want the sponge staff to be spending a large part of their time just playing traffic cop. It’s a job nobody wants to do, and it will negatively impact both their productivity and their morale. Option 3 is a good balance.
One of the concerns I had with this when I was considering it for DBO was that moderators would rely heavily on the flagging and the quality of the manual checks would suffer. This has caused pretty major issues in the past.
Thoughts on allowing any external links but with the use of an interstitial page/modal that tells users “Any file you download from site x has not been checked? Do you wish to continue?”?
Yeah, this is a really big issue and I’m not sure there’s a lot that can be done about it given the presumably voluntary nature of the job.
As I’m sure you are aware, we’ve had a number of cases on DBO where established authors have suddenly released malicious builds. I’d be concerned about this.
Flags can always be evaded and aren’t going to cover every possibility. Reflection and other techniques can fool the checking. I spent a lot of time developing tools for DBO and it ended up with a situation where we had a lot of false positives since there were so many ways to do malicious things.
Would it be possible to have the mod author(s) do some of the certification themselves as part of the approval process? Let them run the tools and provide the reports as part of the submission. Also, would it be beneficial to prioritize the incoming submission queue based on factors such as existing author reputation, availability of source, whether self certify reports are provided, etc?
Personally, I’m for either 1 or 2, but with a modification. Instead of relying on decompilation, force every build to be CI-provided and linked with a commit in a publicly available repository. 3 sounds good at first, but is really useless; having submitted a legitimate plugin is no proof that there isn’t some malicious intent.
Assuming the developer owns/has access to their own CI, what would stop them from replacing the JAR artifact with a malicious one so that users inadvertently downloaded malicious code?
Of course, that would mean Sponge standardizing on a single (or an approved list of) CI providers. For example, Travis shows a link to the applicable commit for each build, so the config could be easily inspected.
I’m leaning towards Option 1 or 3. Preferably review every file similar to Bukkitdev. Unless you guys don’t think you will be able to keep up with the submissions. I can imagine it’s hardly a fun job to do either.
Otherwise Option 3 would be viable as well as long as:
Personally I am more for the 2nd option. I like being able to control my own downloads, be able to have download metrics because the files are being served directly by me.
I’m also not a fan of any system where it requires that you are forced to wait while someone else decompiles your code and tries to read it. In a situation like that you run into the same thing that BukkitDev had where people who didn’t know what they were doing were reading your code and flagging it inappropriately. Either that or you were waiting for long periods of time to get a file reviewed and approved that many times was a critical fix that needed to be out RIGHT THEN but because of overload on the resources (people) available it just wasn’t feasible.
Great idea! I feel like overall there should always be someone reviewing the files manually, because there are always ways to avoid the scanning. Basically it’s about being able to provide the plugin download as quick as possible and the hybrid system mentioned above would actually provide that. So it’s up to the server owner which state of the plugin (unreview/community-reviewed/fully-reviewed) is sufficient.
I honestly think more things should be automated and run by the community instead of a small handful of volunteers. With a proper system in place (some kind of public ratings/reviews) the community can easily determine the good plugins/mods form the malicious ones.
Also, since it is related, I think you should have a look at this thread (especially what I said in post#24): Plugin Annotation MinVersion and Target Fields - Reply #24. Having a system like this would be extremely beneficial to the community, imho.
The plugins should be hosted in a repository-style so plugins can easily rely on other plugins such as WorldEdit’s API or any other plugin’s API without the need for a separate repository.
The problem with a community run system without an approval process is that a file can easily be uploaded that completely violates guidelines and such. There needs to be some sort of approval process in place so the community members using the service can be protected.
I think one issue that Bukkit had was the speed at which plugins were verified. Bukkit was limited by their technology and that they did not have nearly as many workers as would have been best.
I think that opening up the process to the entire community would be an amazing idea, as it would allow for people to assist with things to make everything go smoothly. Anyone who is so much of an a** that they find the need to put back doors in their plugins should not be allowed in the community, and I think that most people have that viewpoint.
I say have the third as it allows for a moderate security and high speeds.
I don’t want to keep helping Curse make money after what they did to me.
BukkitDev staff were all community members. We were not limited by our technology, we spent months developing our own tools to make the process go faster. We never sacrificed security for speed though. We were happy with approvals taking awhile if we weren’t making mistakes. If we got the approval time down to 30 minutes but let through backdoors then we had failed.
See the above response. BukkitDev staff were entirely community members (now paid Curse staff…). Nothing was stopping anyone from applying and we regularly reviewed applications. Putting the burden on everyone would not work. It seems like a novel idea but look at Spigot, they had plugins with backdoors forever that went unnoticed because they are “community reviewed” or something.
Hi. Thanks for calling me incompetent. I hope you can look at this response without a condescending attitude: if you really had to get bug fixes out so fast, 1: do QA before you mark something as release, 2: use a CI service. We only hired competent people to review files. We weren’t perfect, but we knew what we were doing. Files were safe.
This could be a decent compromise.
As long as you have access to the CI backend, you can upload a file and make it look like it came from a certain github commit.
@teozkr your idea simply isn’t feasible for reasons already stated.
Which is why we did completely proactive reviews by hand on BukkitDev. You can’t have speed and security unless you have hundreds of people that are willing, able, and can follow throw on slaving (yes, slaving) away at reviewing files.
It can always happen that someone steals an account of a trusted dev!
There could be a prority based on the plugins popularity. So that important plugins will be checked immidiately.
Also review links to “inofficial downloads” (but allow them), and show an alert when someone wants follow such a link ("You are leaving Sponge Repo. We are not che
I meant no offence in what I said. I just heard from a post that Curse made on the Bukkit forums that the back end technology used on the BukkitDev site was not the best, I wasn’t too sure about its credibility, I guess I was mistaken.
):