Performance evaluation
for Akademia

Comparative report using “Testing of Support Tools for Plagiarism Detection” published by European Network for Academic Integrity

Introduction

  • Akademia was founded in 2016 with the mission to support educational institutions toward improving education and protecting academic integrity. In 2018, Akademia launched the first version of an anti-plagiarism system supporting Albanian and cross-language similarity detection. Cross-lingual similarity detection is a very new and hot topic in the industry, thus, we wanted to be part of an international evaluation of Akademia to understand where we stand in terms of quality and accuracy of our products. As a result, Akademia becomes part of the research that was funded by the European Network for Academic Integrity (ENAI). Our goal was to have a professional and academic evaluation of our product; although, this research was not focused on covering Albanian as a testing language.
  • European Network for Academic Integrity (ENAI) was funded from the European Union and European Commission funds (Erasmus+ Strategic Partnership Projects) to promote academic integrity. Therefore, ENAI has established a research group to test supporting tools in plagiarism detection with a specific focus on digital solutions that offer similarity detection. The adoption of plagiarism detection software is increasing in the education industry, creating the need to conduct proper scientific testing regarding their efficacy in detecting plagiarism.
  • The research and testing of the support tools for plagiarism detection aimed to develop a new methodology where documents used for testing are prepared in different European languages and using different sources. The new methodology is developed and proposed by a group of 9 researchers from different parts of the world and in this research, there are 15 plagiarism detection tools evaluated. The published report analyzes the performance of 15 tools using 8 different languages and methods.
  • Once we have received the initial report from ENAI representatives, Akademia's results were not at its best; however, such results have not been unexpected knowing that we at Akademia have never had the intention to develop the software to support other western languages.
  • This test has been performed using the 2018th version, making us curious to know where we would stand today, as we knew that during 2018-2020 we were making continuous changes and improvements almost every week. Therefore, we decided to re-check all the documents uploaded by the ENAI team. To do so, we needed the help of Ing. Dita Diabolova to guide us on how the testing was performed. Once we got the results from re-submitted documents, it became apparent that Akademia is not a similarity detection software only for Albanian, but for much more.
  • Our aim to publish this report is to share with our stakeholders the updated results based on the methodology used by ENAI. Any institution, university, NGO, or research group interested to validate our data, are more than welcome to contact us and we will be fully transparent and provide you all the necessary resources to conduct an independent evaluation.

Methodology

The research simulated the process of plagiarism checking through a University perspective and based on researchers' first-hand experience with this process, they have prepared a list of ten assumptions that would guide the testing and evaluation.
Assumption 1:
Students tend to plagiarize using documents found on the internet, especially Wikipedia;
Assumption 2:
Some students attempt to disguise their plagiarism;
Assumption 3:
Very few students use advanced techniques for disguising plagiarism (for example, homoglyphs);
Assumption 4:
Most plagiarizing students do not submit a complete plagiarism from one source, but use multiple sources;
Assumption 5:
Instructors generally have many documents to test at one time;
Assumption 6:
There are legal restrictions on instructors submitting student work;
Assumption 7:
In some situations the instructor only reviews the reports, submission is done either by the students themselves or by a teaching assistant
Assumption 8:
Instructors do not have much time to spend on reviewing reports;
Assumption 9:
Reports must be stored in printed form in a student's permanent record if a sanction is levied;
Assumption 10:
Universities wish to know how expensive the use of a system will be on a yearly basis;

Methodology

Documents prepared to evaluate all systems were written in eight different languages to understand which systems are useful for each language.
Qekisht
Czech
Gjermanisht
German
Letonisht
Latvian
Spanjollisht
Spanish
Anglisht
English
Italisht
Italian
Sllovakisht
Slovak
Turqisht
Turkish
Qekisht
Czech
Anglisht
English
Gjermanisht
German
Italisht
Italian
Letonisht
Latvian
Sllovakisht
Slovak
Spanjollisht
Spanish
Turqisht
Turkish

Methodology

Documents used for testing were based on different types of sources of information and plagiarism methods used mostly by students in their writings.

Methodology

Sources used in every document have been tested in various versions based on the most widely used methods by students in their writings.

Methodology

For each criterion, the evaluation used is based on the accuracy systems reported plagiarism. In every criterion, there is a rating system where 0 points being the poorest and 5 points mean the best..

Methodology

This methodology is designed by the best researchers in this field, making it the only credible methodology available in the science community. The aim of this methodology is to be used in evaluating the two major factors important for any tool to be effective; coverage and usability. Coverage practically means evaluating the quality and accuracy of similarity findings using different languages, sources, methods, and scenarios..

Methodology

On the other hand, the term usability is used to represent the evaluation of the functionalities enabling users to enjoy a friendlier and convenient interaction.

Below we will present you, comparative data between the 2018 version (Beta) results and 2020 version (Release candidate) results. This comparative analysis is clear evidence that during the last two years we have made great progress.

Coverage

In this part, we will present results on evaluated factors such as language coverage, plagiarism methods, types of plagiarism sources, and scenarios that researchers deemed most widely used by students in their writings.
Language coverage
2018
In the published report, using the 2018 version, Akademia’s highest rating was in the English language with 2.9 points and the lowest rating in Slovakian with 0.8 points. Italian had not been tested at all in 2018 due to the fact that we have not provided the research group with the option to select the Italian language. Using ranking as a metric, it can be seen that Akademia’s best rank was in Turkish language ranking as 4th while in other languages our rank was between 8 and 15.

Languages

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Czech 1.4 1.5 1.1 0.9 0.0 0.7 1.8 1.5 1.5 1.5 2.2 1.6 1.0 1.9 1.0
German 2.2 1.6 2.3 0.3 1.3 0.7 3.3 2.5 1.6 2.6 2.6 3.2 2.1 2.4 1.9
English 2.9 3.3 2.8 0.9 0.9 1.3 3.2 2.1 1.2 4.0 3.0 3.7 3.2 3.3 1.3
Spanish 2.0 2.1 2.4 0.2 0.0 0.6 2.7 2.0 2.7 2.8 2.6 2.8 2.6 3.4 2.1
Italian - 2.9 2.7 0.1 0.0 0.0 3.0 3.7 1.3 3.7 3.7 3.6 3.0 2.6 2.9
Latvian 1.5 2.3 1.8 0.6 0.0 0.0 2.3 3.8 1.6 2.7 2.6 1.2 2.3 3.2 2.3
Slovak 0.8 2.0 1.7 0.9 0.0 1.0 2.1 2.0 1.5 1.3 1.6 1.7 1.7 2.6 1.3
Turkish 2.0 1.5 1.3 0.3 0.5 1.0 2.1 1.7 1.1 1.9 1.8 2.0 1.4 3.1 1.2

Rank

Akademia
Czech 10
German 8
English 8
Spanish 11
Italian 15
Latvian 11
Slovak 14
Turkish 4
Table 1: Akademia's score and rank for every language (Beta version - 2020)
2020
After our revaluation using the latest version, performance based on the same documents improves drastically. Akademia’s highest score is English with 3.9 points while again Slovakian has the lowest score of 1.1 points. The progress can be seen as Akademia ranks 1st in Spanish and Italian, 2nd in Czech, English, and Turkish, 3rd in German, 4th in Latvian, and 12th in Slovakian.

Languages

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Czech 2.0 1.5 1.1 0.9 0.0 0.7 1.8 1.5 1.5 1.5 2.2 1.6 1.0 1.9 1.0
German 3.0 1.6 2.3 0.3 1.3 0.7 3.3 2.5 1.6 2.6 2.6 3.2 2.1 2.4 1.9
English 3.9 3.3 2.8 0.9 0.9 1.3 3.2 2.1 1.2 4.0 3.0 3.7 3.2 3.3 1.3
Spanish 3.4 2.1 2.4 0.2 0.0 0.6 2.7 2.0 2.7 2.8 2.6 2.8 2.6 3.4 2.1
Italian 3.8 2.9 2.7 0.1 0.0 0.0 3.0 3.7 1.3 3.7 3.7 3.6 3.0 2.6 2.9
Latvian 2.6 2.3 1.8 0.6 0.0 0.0 2.3 3.8 1.6 2.7 2.6 1.2 2.3 3.2 2.3
Slovak 1.1 2.0 1.7 0.9 0.0 1.0 2.1 2.0 1.5 1.3 1.6 1.7 1.7 2.6 1.3
Turkish 3.0 1.5 1.3 0.3 0.5 1.0 2.1 1.7 1.1 1.9 1.8 2.0 1.4 3.1 1.2

Rank

Akademia
Czech 2
German 3
English 2
Spanish 1
Italian 1
Latvian 4
Slovak 12
Turkish 2
Table 2: Akademia's score and rank for every language (Release candidate - 2020)
Language subgroups
2018
Based on language coverage results, researchers calculated an average score for three language subgroups: Germanic, Roman and Slavic. Akademia had the highest rating with 2.5 points in Germanic languages and the lowest with 1.1 points in Slavic languages. Researchers were unable to generate a score for Romanic language as a result of Italian not being tested. In ranking terms, Akademia was ranked 8th in Germanic, 12th in Slavic and 15th in Roman languages.

Language subgroups

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Germanic 2.5 2.5 2.5 0.6 1.1 1.0 3.3 2.3 1.3 3.3 2.8 3.5 2.6 2.9 1.6
Roman - 2.5 2.6 0.2 0.0 0.3 2.8 2.9 2.0 3.3 3.2 3.2 2.8 3.0 2.5
Slavic 1.1 1.8 1.4 0.9 0.0 0.8 1.9 1.8 1.5 1.4 1.9 1.6 1.4 2.3 1.2

Rank

Akademia
Germanic 8
Roman 15
Slavic 12
Table 3: Akademia's score and rank on language subgroups (Beta version - 2018)
2020
However, in the 2020 revaluation, there is a huge improvement as Akademia scores the highest in Roman languages with 3.7 points and Slavic with 3.5 points. In Germanic languages, we were able to improve only by 0.3 points and we scored 2.8 points. From a ranking perspective, Akademia ranked 1st in Slavic and Roman languages and 4th in Germanic languages.

Language subgroups

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Germanic 2.8 2.5 2.5 0.6 1.1 1.0 3.3 2.3 1.3 3.3 2.8 3.5 2.6 2.9 1.6
Roman 3.7 2.5 2.6 0.2 0.0 0.3 2.8 2.9 2.0 3.3 3.2 3.2 2.8 3.0 2.5
Slavic 3.5 1.8 1.4 0.9 0.0 0.8 1.9 1.8 1.5 1.4 1.9 1.6 1.4 2.3 1.2

Rank

Akademia
Germanic 4
Roman 1
Slavic 1
Table 4: Akademia's score and rank in language subgroups (Release candidate - 2020)
Types of plagiarism sources
2018
In this part, researchers have assessed the performance of the systems based on the different sources where the students are influenced in their writings. Based on 2018 results, Akademia’s highest score was on the finding similarities from Wikipedia, with a rating of 2.3 points, while the lowest was on the categories of the Student Thesis, with a rating of 0.2 points. Akademia ranked 9th on Wikipedia, 6th on OA Papers, 7th on Student thesis, and 13th on online articles.

Type of source

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Wikipedia 2.3 2.3 2.2 0.9 0.8 0.9 2.4 2.8 1.5 2.4 2.6 2.6 2.3 3.0 2.0
OA Paper 1.4 1.1 1.3 0.4 0.4 0.5 1.6 1.1 1.2 1.5 1.2 1.8 1.5 1.7 0.8
Student thesis 0.1 0.1 0.1 0.1 0.1 1.0 0.3 0.1 1.3 0.3 2.1 0.1 0.1 0.8 0.1
Online articles 0.6 2.2 1.8 0.7 0.5 0.6 1.8 2.8 1.4 2.5 2.3 1.6 2.2 2.0 2.0

Rank

Akademia
Wikipedia 9
OA Paper 6
Student thesis 7
Online articles 13
Table 5: Akademia's score and rank on plagiarism sources (Beta version - 2018)
2020
In the 2020 version, Akademia’s performance improves as a result of cooperating with third-party libraries and archives, accompanied by improvements in crawling algorithms. These measures enabled Akademia to perform as the best tool in finding similarities from Wikipedia, with a rating of 3.0 points, but yet again we rate poorly in cases where copying is based on student thesis with a rating of 0.1 points. In 2020, Akademia ranks 1st on Wikipedia, 2nd on OA Papers, 7th on Student thesis, and 3rd on online articles.

Type of source

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Wikipedia 3.0 2.3 2.2 0.9 0.8 0.9 2.4 2.8 1.5 2.4 2.6 2.6 2.3 3.0 2.0
OA Paper 1.7 1.1 1.3 0.4 0.4 0.5 1.6 1.1 1.2 1.5 1.2 1.8 1.5 1.7 0.8
Student thesis 0.1 0.1 0.1 0.1 0.1 1.0 0.3 0.1 1.3 0.3 2.1 0.1 0.1 0.8 0.1
Online article 2.4 2.2 1.8 0.7 0.5 0.6 1.8 2.8 1.4 2.5 2.3 1.6 2.2 2.0 2.0

Rank

Akademia
Wikipedia 1
OA Publikime 2
Student thesis 7
Online article 3
Table 6: Akademia's score and rank on plagiarism sources (Release candidate - 2020)
Plagiarism methods
2018
Among many methods used for copying others’ written work, the researchers have analyzed four most often used methods for copying; them being, copy&paste, usage of synonyms, bad paraphrasing, and translation. According to the version of 2018, Akademia has performed best in copy&paste, with a rating of 2.1 points, and the lowest in translation, with a rating of 1.1 points. However, it’s worth noting that the researchers have evaluated Akademia as the only platform performing semantic analysis and making possible the comparison of documents through translating. Other systems had a much lower performance in translating except Akademia as quoted in the study;
“The worst performance of the systems in this test was indeed the translation plagiarism, with one notable exception—Akademia”

Plagiarism Methods

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Copy&Paste 2.1 2.7 2.8 1.1 0.8 1.0 2.9 2.6 2.6 2.7 3.1 2.8 3.1 3.2 2.6
Synonyms 1.8 2.3 1.9 0.7 0.6 0.8 2.2 2.7 1.2 2.5 2.5 2.5 2.1 2.7 1.6
Paraphrasing 1.3 1.2 1.0 0.4 0.5 0.7 1.4 1.7 0.8 1.5 1.3 1.5 1.1 1.6 0.9
Translation 1.1 0.0 0.1 0.0 0.2 0.3 0.2 0.2 0.3 0.0 0.4 0.3 0.2 0.5 0.3

Rank

Akademia
Copy&Paste 12
Synonyms 10
Paraphrasing 6
Translation 1
Table 7: Akademia's score and rank on plagiarism methods (Beta version - 2018)
2020
Comparing the 2018 to 2020 version, on plagiarism methods, we see huge improvements between the two versions. This is a result of ongoing work and quality assurance; something we at Akademia give special attention to. After the revaluation, Akademia has performed best in copy&paste with a rating of 2.8 points, and the lowest in Paraphrasing, with a rating of 1.9 points. Our focus for cross-lingual similarity detection had made possible that from a rating of 1.1 points for the 2018 version, we got up to a rating of 2.0 points for the 2020 version. At the same time, an evaluation of 2.0 points on translating, top’s Akademia as the only system worldwide that offers the possibility of cross-lingual plagiarism detection. It is worth mentioning, Akademia not only improved in translation but is also ranked 4th in copy&paste, 2nd on synonyms, 1st in paraphrasing.

Plagiarism Methods

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Copy&Paste 2.8 2.7 2.8 1.1 0.8 1.0 2.9 2.6 2.6 2.7 3.1 2.8 3.1 3.2 2.6
Synonyms 2.5 2.3 1.9 0.7 0.6 0.8 2.2 2.7 1.2 2.5 2.5 2.5 2.1 2.7 1.6
Paraphrasing 1.9 1.2 1.0 0.4 0.5 0.7 1.4 1.7 0.8 1.5 1.3 1.5 1.1 1.6 0.9
Translation 2.0 0.0 0.1 0.0 0.2 0.3 0.2 0.2 0.3 0.0 0.4 0.3 0.2 0.5 0.3

Rank

Akademia
Copy&Paste 4
Synonyms 2
Paraphrasing 1
Translation 1
Table 8: Akademia's score and rank on plagiarism methods (Release candidate - 2020)
Single-source vs. Multi-source Documents
2018
Researchers used the scenario when the documents contain plagiarism from only one source and the scenario when the documents contain plagiarism from multiple sources. Both scenarios were compared based on different sources (Table 9) and Wikipedia (Table 10).
According to the 2018 evaluations, Akademia has performed as the very best at the one-source scenario, with a rating of 2.0 points, but poor at the multiple source scenario, with a rating of 1.6 points. Such ratings indicate that Akademia ranked as 10th in a single-source scenario and 11th in the multi-source scenario.

All sources

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Single-source 2.0 2.2 2.0 0.5 0.5 0.6 2.3 2.5 1.5 2.4 2.5 2.5 2.3 2.7 1.7
Multi-source 1.6 1.7 2.0 0.6 0.2 0.8 3.3 2.0 1.7 2.9 2.5 2.5 1.8 3.1 1.5

Rank

Akademia
Single-source 10
Multi-source 11
Table 9: Akademia's score and rank on single-source vs. Multi-source using all sources (Beta version - 2018)
2018
A single evaluation was made for the findings using articles from Wikipedia, to evaluate the performance according to one-source scenarios and multiple-source scenarios.
Based on this approach, Akademia’s highest rating was on the one-source scenario of 2.4 points; yet the lowest on the multiple-source scenario with a rating of 1.8 points. Akademia’s rank in terms of results using Wikipedia for the single-source scenario was 9th and 12th in the multi-source scenario.

All sources

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Single-source 2.4 2.5 2.3 0.8 0.7 0.9 2.5 2.9 1.6 2.6 2.7 2.8 2.5 3.2 2.1
Multi-source 1.8 2.4 2.2 1.3 0.8 0.8 3.6 3.3 2.0 3.3 3.2 3.4 1.9 3.6 2.3

Rank

Akademia
Single-source 9
Multi-source 12
Table 10: Akademia's score and rank on single-source vs. Multi-source using all sources (Beta version - 2018)
2020
On the 2020 revaluation of Akademia, based on the criteria above, we see significant progress. The results of this revaluation for all the sources according to the one-source scenario, Akademia has a rating of 3.0 points; whereas in the multi-source scenario scored a rating of 2.3 points. Rankings improved drastically, where Akademia is ranked 1st in single-source and 5th in multi-source.

All sources

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Single-sources 3.0 2.2 2.0 0.5 0.5 0.6 2.3 2.5 1.5 2.4 2.5 2.5 2.3 2.7 1.7
Multi-source 2.3 1.7 2.0 0.6 0.2 0.8 3.3 2.0 1.7 2.9 2.5 2.5 1.8 3.1 1.5

Rank

Akademia
Single-sources 1
Multi-source 5
Table 11: Akademia's score and rank on single-source vs. multi-source using all sources (Release candidate - 2020)
2020
Regarding the analysis where Wikipedia was used to compare the performance of the systems on the one-source and multiple-source aspect, Akademia is rated as the best with 3.3 points on the one-source scenario, while on the multiple-source scenario, is rated 2.7 points. Similar to the previous case, Akademia ranks 1st on a single-source scenario using Wikipedia and 5th on a multi-source scenario using Wikipedia.

All sources

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper
Single-sources 3.3 2.5 2.3 0.8 0.7 0.9 2.5 2.9 1.6 2.6 2.7 2.8 2.5 3.2 2.1
Multi-source 2.7 2.4 2.2 1.3 0.8 0.8 3.6 3.3 2.0 3.3 3.2 3.4 1.9 3.6 2.3

Rank

Akademia
Single-sources 1
Multi-source 5
Table 12: Akademia's score and rank in single-source vs. multi-source using Wikipedia (Release candidate - 2020)

Usability

Usability was analyzed as a necessity that the systems must offer the possibility for users to control documents easier. Researchers, in this case, have analyzed 23 objective criteria to evaluate the aspect of usability for each system. The evaluation criteria are grouped into three categories such as workflow process, result presentation, and other aspects.
Workflow process
Is it possible to upload and test multiple documents at the same time? Does the system ask to fill in metadata for documents? Does the system use original file names for the report? Is there any word limit for the document testing? Does the system display text in the chosen language only? Can the system process large documents (for example, bachelor thesis)?
Result presentation
Reports are downloadable. Results are saved in the user's account and can be reviewed afterwards. Matched passages are highlighted in the online report. Matched passages are highlighted in the downloaded report (offline). Evidence of similarity is demonstrated side-by-side with the source in the online report. Evidence of similarity is demonstrated side-by-side with the source in the downloaded report. Document formatting is not changed in the report. Document page numbers are shown in the report. The report is not spoiled by false positives.
Other aspects
System costs are clearly stated in the system homepage. Information about a free system trial version is advertised on the webpage. The system can be integrated as an API to a learning management system. The system can be integrated with the Moodle platform. The system provides call support. The call support is provided in English. English is properly used on the website and reports. There are no external advertisements.
Usability 2018 vs. 2020
2018
The usability evaluation aspect is relatively simple. Each of the 23 criteria is rated at a value of either 1 (positive) or 0 (negative). Akademia’s evaluation for the 2018 version has been rated 2 out of 6 points regarding the process criteria, 5 out of 9 points on the presentation of the results criteria, and 2 out of 8 points for the other aspects.

Workflow process

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper Total
Upload multiple documents 0 0 1 0 1 0 0.5 1 0 1 0 1 1 1 1 8.5
No metadata required 0 1 1 1 1 0 1 1 1 1 0 0 1 1 0 10
Original filenames 0 0 1 0 1 1 1 0 0 1 1 1 1 1 1 10
No word limit 1 0 1 0 1 1 1 1 0 1 1 1 1 1 1 12
Text in chosen language 0 1 1 1 1 0 0 1 1 1 1 1 1 1 1 13
Large documents 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 13
Total 2 3 6 2 6 3 4.5 5 2 6 4 5 6 6 5 4 \ 11
Table 13: Akademia's score in process component of usability (Beta version - 2018)

Presentation of results

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper Total
Downloadable report 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 13
Results saved 1 0 1 0 1 1 1 1 0 1 1 1 1 1 1 12
Highlights text match online 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 13
Highlights text matched offline 0 0 1 0 1 0 1 1 1 1 1 1 1 1 1 11
Side-by-side comparison online 0 0 0 1 1 0 0.5 0 0 1 0 0 0 1 0 4.5
Side-by-side comparison offiline 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1
Document formatting not changed 1 0 1 0 0 0 0 0 0 1 0 1 1 0 0 5
Page numbers shown 1 0 1 0 0 0 0 0 0 1 1 1 1 1 1 8
No false positives 0 1 1 1 1 0 1 1 0 1 1 1 0 1 1 11
Total 5 1 6 3 6 3 5.5 5 3 8 6 7 6 8 6 5 \ 9
Table 14: Akademia's score in result presentation component of usability (Beta version - 2018)

Other aspects

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper Total
Costs clearly stated 0 1 1 1 0 0 1 1 1 1 1 0 1 0 1 10
Free trial advertised on web page 0 1 0 1 0 0 1 1 1 1 1 0 1 0 0 8
API Integration 0 1 1 0 0 0 1 1 0 1 1 1 1 1 0 9
Moodle Integration 0 0 0 0 0 0 1 1 0 1 1 1 1 1 0 7
Has call support 0 0 1 0 0 0.5 0.5 1 0.5 1 0.5 0.5 1 1 0 7.5
Support speaks english 0 0 1 0 0 0.5 0 1 1 0 0 1 1 1 0 6.5
No grammatical mistakes 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
No external advertisments 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 14
Total 2 5 6 3 2 3 6.5 8 5.5 7 6.5 5.5 8 6 3 5 \ 10
Table 15: Akademia's score in other aspects component of usability (Beta version - 2018)
2020
Platforma Akademia in 2020 does not have a significant difference with the one in 2018. However, the changes made give users the possibility to choose the language in which the document is written, and the “false positives” are almost eliminated. Akademia also offers the possibility for API Integration and continuous technical support through live-chat or direct phone calls, in both Albanian and English language.

Workflow process

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper Total
Upload multiple documents 0 0 1 0 1 0 0.5 1 0 1 0 1 1 1 1 8.5
No metadata required 0 1 1 1 1 0 1 1 1 1 0 0 1 1 0 10
Original filenames 0 0 1 0 1 1 1 0 0 1 1 1 1 1 1 10
No word limit 1 0 1 0 1 1 1 1 0 1 1 1 1 1 1 12
Text in chosen language 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 13
Large documents 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 13
Total 3 3 6 2 6 3 4.5 5 2 6 4 5 6 6 5 4 \ 11
Table 16: Akademia's score in process component of usability (Release candidate - 2020)

Result presentation

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper Total
Downloadable report 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 13
Results saved 1 0 1 0 1 1 1 1 0 1 1 1 1 1 1 12
Highlights text match online 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 13
Highlights text match offline 0 0 1 0 1 0 1 1 1 1 1 1 1 1 1 11
Side-by-side comparison online 0 0 0 1 1 0 0.5 0 0 1 0 0 0 1 0 4.5
Side by side comparison offiline 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1
Document formatting not changed 1 0 1 0 0 0 0 0 0 1 0 1 1 0 0 5
Page numbers shown 1 0 1 0 0 0 0 0 0 1 1 1 1 1 1 8
No false positives 1 1 1 1 1 0 1 1 0 1 1 1 0 1 1 12
Total 6 1 6 3 6 3 5.5 5 3 8 6 7 6 8 6 5 \ 9
Table 17: Akademia's score in results presentation component of usability (Release candidate - 2020)

Other aspects

Akademia Copyscape Docol©c Dupli Checker DPV intihal.net PlagAware PlagiarismCheck.org Plagiarism Software PlagScan StrikePlagiarism.com Turnitin Unicheck Urkund Viper Total
Costs clearly stated 0 1 1 1 0 0 1 1 1 1 1 0 1 0 1 10
Free trial advertised on web page 0 1 0 1 0 0 1 1 1 1 1 0 1 0 0 8
API Integration 1 1 1 0 0 0 1 1 0 1 1 1 1 1 0 10
Moodle Integration 0 0 0 0 0 0 1 1 0 1 1 1 1 1 0 7
Has call support 1 0 1 0 0 0.5 0.5 1 0.5 1 0.5 0.5 1 1 0 8.5
Support speaks English 1 0 1 0 0 0.5 0 1 1 0 0 1 1 1 0 7.5
No grammatical mistakes 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
No external advertisments 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 14
Total 5 5 6 3 2 3 6.5 8 5.5 7 6.5 5.5 8 6 3 5 \ 10
Table 18: VAkademia's score in other aspects component of usability (Release candidate - 2020)

Coverage and Usability

2018
After every system has been evaluated in different criteria, researchers have been able to better understand where each system is positioned in terms of coverage and usability.
To illustrate how these systems performed, a 2-dimensional graph has been used where the X-axis depicts coverage and Y-axis depicts usability. The overall rating of Akademia in terms of coverage is 1.96 out of 5.00 points while the usability rating is 9 out of 23 points.
Overall ranking for Akademia on the aspect of coverage and usability

Graph 1: Akademia's positioning based on Coverage and Usability results (Beta version - 2018)
2020
On the other hand, in 2020, Akademia’s overall score in coverage is 2.82 while the overall rating in usability is 14 out of 23. Therefore, Akademia ranks the 2nd best performing system in coverage; whereas, in terms of usability aspect, Akademia ranks 11th.
Overall ranking for Akademia on the aspect of coverage and usability

Graph 2: Akademia's poisitioning based on Coverage and Usability results (Release candidate - 2020)
Coverage and Usability Performance
Researchers have classified tools used for testing based on their performance regarding coverage and usability. For each of two categories, tools have been classified as useful systems, partially useful systems, marginally useful systems, or unsuited for academic institutions.
Ilustration 1: The four performance classifications of tools
Coverage 2018 vs. 2020
Based on results achieved in every Coverage criteria, Akademia in 2018 was classified as a “marginally useful system" with an overall rating of 1.96 points. After revaluation in 2020, Akademia is classified as a "partially useful system" with an overall rating of 2.82 points.
It is worth mentioning that none of the systems is classified as a “useful system”.
Ilustration 2: Akademia's coverage performance in 2018 and 2020
Usability 2018 vs. 2020
Akademia did not progress in terms of Usability from 2018 to 2020. As shown in Illustration 2, Akademia in both 2018 and 2020 is classified as a "marginally useful system".
Ilustration 3: Akademia's usability performance in 2018 vs. 2020
Akademia is one of the most advanced tools in the world in terms of similarity detection and the only tool offering cross-lingual similarity detection!

Conclusions

Akademia was evaluated by the group of ENAI researchers, using Akademia’s beta version released in 2018. Based on these evaluations, we were marginally above the recommended minimum of offering services for Universities. However, after the revaluation results, using the updated version of Akademia, it is fair to say that Akademia is one of the best systems available for many reasons. It’s worth mentioning that in 2018, as well as in 2020, we are the only tool offering cross-lingual similarity detection.
We are deeply honored and thankful to be working with university partners in Kosovo and Albania as they are the true reason we have been able to innovate. All we can say is that we have, we are, and we will be listening closely to your needs and recommendations as this is the only way to learn how far we can go. This achievement of ours enables first-time to exist, therefore, thank you!
Akademia thanks the European Network for Academic Integrity and the working group of:
  • Tomáš Foltýnek,
  • Dita Dlabolová,
  • Alla Anohina-Naumeca,
  • Salim Razı,
  • Július Kravjar,
  • Laima Kamzola,
  • Jean Guerrero-Dib,
  • Özgür Çelik,
  • Debora Weber-Wulff
for their work and contribution in publishing a clear and resourceful methodology beneficial for both; companies and universities.
Europian Network for Academic Integrity
akademia logo
https://www.akademia.al/ info@akademia.al +383 44 112 596 Ahmet Krasniqi C3/7 H3, nr.39/40, Prishtina 10000, Kosova
akademia logo
To read the report using your smartphone, please download the report in PDF format below.