Welcome to Judgels’ documentation!¶

Applications¶

Judgels consists of several applications that work with each other. Each application has a codename after a Greek archangel name.

At the moment, there are five applications in Judgels:

1. Single Sign-On (Jophiel): authenticating and authorizing users.
2. Repository Gate (Sandalphon): storing programming problems and materials.
3. Competition Gate (Uriel): holding programming contests.
4. Middleware (Sealtiel): providing message queues and transmissions between applications.;
5. Grader Application (Gabriel): grading programming submissions.

All applications don’t have to be installed in one machine. We can install an application in one machine and the others in other machines. This ensures good scalability.

We are still working on other applications for programming courses and online judges.

License¶

Judgels is licensed using GNU GPL version 2.

Contributions¶

Currently, the project is maintained by (in alphabetical order) Ashar Fuadi and Jordan Fernando. Public contributions are welcome. Please consult the Developer’s Guide on how Judgels works in details.

Requirements¶

The following packages must be installed on your machine(s), for all Judgels applications:

• Oracle Java 8
• MySQL >= 5.6
• SBT >= 0.13
• Git;
• Nginx (for reverse proxy)

All Judgels applications can run on any operating systems, except for Gabriel.

Here are the requirements for each specific Judgels application.

Installation¶

To install judgels, you need to clone the latest version of judgels from GitHub and run “scripts/pull-latest.sh” to clone all of its’ submodules.

After cloning judgels from GitHub, you can run each applications from their directory by executing “activator” or “sbt” (only for gabriel and gabriel-commons). From the submodules, the directories with suffix “_commons” are to reduce codes duplication. Please refer to Play Framework and SBT documentation to run the applications.

Configuration¶

After installing judgels, you need to fill the configuration such as database, session cookie name, etc. There are default configuration files in “conf” directory or “src/resources/conf” (for Gabriel) with suffix “_default.conf”. You can copy the configuration files, remove the suffix “_default” (i.e. application_default.conf -> application.conf) and configure the values accordingly.

Configuration¶

First copy default configuration in “conf” directory: - application_default.conf -> application.conf - db_default.conf -> db.conf

Create a database for storing Jophiel data. Then change the configuration of Jophiel to point to the database in “conf/db.conf” file.

You also need to configure smtp email to support Jophiel email verification feature. You can find the configuration in “conf/application.conf” file.

Jophiel needs a directory to save data such as avatar files. You need to create one and configure the directory in “conf/application.conf” file.

Jophiel depends on judgels-play-commons to run, make sure judgels-play-commons is on the same level of directory with Jophiel.

Running Jophiel¶

You can run Jophiel by using “activator start” command. By default, it will listen on port 9000. You can access it via web browser using url “http://localhost:9000”.

At the first run, you need to register and verify your email in order to login. After your first user is created, you can change the value of your roles in table “jophiel_user” from “user” into “user,admin” to access full feature.

Jophiel User Roles¶

Currently there are two roles in Jophiel:

1. Normal User

   Currently normal user can only edit their profile on Jophiel.

2. Admin

   Admin can manage (CRUD) users and manage (CRUD) clients. Jophiel clients can use Jophiel single sign on for authentication and authorization.

Single Sign On Protocol¶

Jophiel uses Open Id Connect Protocol for single sign on. Since Open Id Connect is built on top of OAuth 2.0 protocol, it provides authorization besides authentication. Each Jophiel Client’s manage their own authentication and authorization using Jophiel’s provided API, access token, and id token.

Future Target¶

Currently Jophiel is meant to be used for private clients of IA TOKI. In the future, we want each user to be able to use Jophiel Single Sign On to create applications.

Configuration¶

First copy default configuration in “conf” directory: - application_default.conf -> application.conf - db_default.conf -> db.conf

Create a database for storing Sandalphon data. Then change the configuration of Sandalphon to point to the database in “conf/db.conf” file.

Sandalphon uses Jophiel for authentication and authorization. You need to configure Jophiel’s parameter in “conf/application.conf” file.

Sandalphon needs a directory to save data such as problems and submissions. You can find the configuration in “conf/application.conf” file.

Sandalphon uses Gabriel to grade programming submissions. Gabriel can only be contacted through Sealtiel that acts as queue manager and router. You need to configure the parameter in “conf/application.conf” file.

Sandalphon depends on judgels-play-commons, judgels-gabriel-commons, and judgels-frontend-commons to run, make sure judgels-play-commons, judgels-gabriel-commons, and judgels-frontend-commons are on the same level of directory with Sandalphon.

Running Sandalphon¶

You can run Sandalphon by using “activator start” command. By default, it will listen on port 9000. You can access it via web browser using url “http://localhost:9000”.

At the first run, you need to have Jophiel’s account in order to login. After you login into Sandalphon you can change the value of your roles in table “sandalphon_user_role” from “user” into “user,writer,admin” to access full feature.

Sandalphon User Roles¶

Currently there are three roles in Sandalphon:

1. Normal User

   Currently normal user who can do nothing on Sandalphon.

2. Writer

   Writer can manage (CRUD), test programming problems and give programming problem access to clients.

3. Admin

   Admin can manage (CRUD) clients and graders. Sandalphon’s clients can render programming problem that has been given access to. Sandalphon’s graders can fetch testcases through given API.

Client’s Problem Rendering¶

After Client registration, each Client is given client ID and client Secret for Sandalphon’s API authentication. Currently, Sandalphon only has the API to render problem.

Each Problem can be given access to any client registered on Sandalphon. Client that has been given access gets “Problem Secret”. Client can render problem using certain API that requires Time-based One Time Password that can be generated using “Problem Secret”. TOTP is used to ensure security that programming problem can only be accessed through the link in limited duration.

Future Target¶

Currently Sandalphon’s writer are only accessible for limited people. In the future, we want every user to become writer that can create problem and share it with each others.

Blackbox Grading¶

In blackbox grading, user’s solution is run with provided input testcase and evaluated based on the output of the solution. Blackbox grading don’t evaluate the structure or style of the code. Blackbox grading also evaluate the solution performance by setting constraints such as time and memory limit.

Constraints¶

User solution is run in a isolated environment with some constraints. Isolation of user solution is done by using sandbox. Currently, we are using isolate as sandbox.

Evaluators¶

Below are the terms that are used in Judgel’s evaluators:

Problem Types¶

Therea re three types of programming problem:

Configuration¶

Install RabbitMQ on your Operating System.

First copy default configuration in “conf” directory: - application_default.conf -> application.conf - db_default.conf -> db.conf

Create a database for storing Sealtiel data. Then change the configuration of Sealtiel to point to the database in “conf/db.conf” file.

Sealtiel depends on judgels-play-commons to run, make sure judgels-play-commons is on the same level of directory with Sealtiel.

Running Sealtiel¶

You can run Sealtiel by using “activator start” command. By default, it will listen on port 9000. You can access it via web browser using url “http://localhost:9000”.

Message Queue¶

Message delivery in Sealtiel utilize message queue. Sealtiel uses RabbitMQ to contains the queue. In Judgels, Sealtiel acts as router to connect components with graders. Components and graders in Judgels have to be registered as client of Sealtiel to utilize the message delivery.

Polling¶

In message delivery, Sealtiel acts as passive components. Sealtiel only accept incoming message delivery request from Client and put it into queue. In order for Sealtiel’s client to get the message, they must actively poll message from Sealtiel.

After polling the message from Sealtiel, the client need to send ack to notify if the message has been processed. If the message hasn’t been processed after some times, the message will be requeued. Client can extend the acknowledgement time limit using Sealtiel’s API.

Grading Distribution¶

Message queue in Sealtiel can be used for grading distribution purposes. Currently, each client is associated with one queue. Client can send message to each other through the queue identity. To distribute grading, all grader instance can be registered as one client and obtains one queue. All grading request message can be sent to graders’ client queue. Since each grader poll from the queue every interval time, load balancer is created naturally.

Future Target¶

Currently all grader are registered as one client. There need to be implementation of a group of client (channel) to register each grader as one client and support more routing options.

Configuration¶

Install SBT on your Operating System.

First copy default configuration in “src/main/resources/conf” directory: - application_default.conf -> application.conf

Gabriel need to connect to Sealtiel to fetch grading requests. You need to configure the parameter in “src/main/resources/conf/application.conf” file.

Gabriel need to connect to Sandalphon to fetch problem evaluator files (testcases, subtask configuration, constraints, checkers, etc). You need to configure the parameter in “src/main/resources/conf/application.conf” file.

Next, you have to configure Moe. Run ./configure and make in judgels-moe repository. Gabriel uses Moe to grade submissions. You need to configure the parameter in “src/main/resources/conf/application.conf” file.

Gabriel depends on judgels-moe and judgels-gabriel-commons to run, make sure they are on the same level of directory with Gabriel.

Running Gabriel¶

You can run Gabriel by using “sbt” command to enter sbt console and enter “run” command in the console.

Grading Engine¶

Gabriel can grade types of submissions based on existing grading engines. Currently there are three grading engines as described on Problem Types:

• Batch
• Batch with Subtasks
• Interactive with Subtasks

Future Target¶

Currently Gabriel’s grading engine are coded along with Gabriel. In the future, Gabriel’s grading engine are supposed to be separated as add-on that can be fetch by Gabriel.

Configuration¶

First copy default configuration in “conf” directory: - application_default.conf -> application.conf - db_default.conf -> db.conf

Create a database for storing Uriel data. Then change the configuration of Uriel to point to the database in “conf/db.conf” file.

Uriel uses Jophiel for authentication and authorization. You need to configure Jophiel’s parameter in “conf/application.conf” file.

Uriel needs a directory to save data such as team avatars. You can find the configuration in “conf/application.conf” file.

Uriel uses Sandalphon to render problems. You need to configure Sandalphon’s parameter in “conf/application.conf” file.

Uriel uses Gabriel to grade programming submissions. Gabriel can only be contacted through Sealtiel that acts as queue manager and router. You need to configure the parameter in “conf/application.conf” file.

Uriel depends on judgels-play-commons, judgels-gabriel-commons, and judgels-frontend-commons to run, make sure judgels-play-commons, judgels-gabriel-commons, and judgels-frontend-commons are on the same level of directory with Uriel.

Running Uriel¶

You can run Uriel by using “activator start” command. By default, it will listen on port 9000. You can access it via web browser using url “http://localhost:9000”.

At the first run, you need to have Jophiel’s account in order to login. After you login into Uriel you can change the value of your roles in table “uriel_user_role” from “user” into “user,admin” to access full feature.

Uriel User Roles¶

Currently there are two main roles in Uriel:

1. Normal User

   Normal user can register to open contests, enter registered contests, solve problems, and do other contest related stuff.

3. Admin

   Admin can manage (CRUD) contests and user roles.

Besides main roles, there are also implicit roles in Uriel:

1. Coach

   Coach can start a virtual contest for the team. This role is specifically made to be used in APIO like contest.

2. Supervisor

   Supervisor can manage contest announcements, clarifications, contestants, problems, and submissions based on the access given.

3. Manager

   Manager is assigned by admin. Manager can do stuff that supervisor can do and manage supervisors and their access.

Type¶

Currently, there are two contest types:

• Standard

  Standard contest are normal contest with fixed duration and user can only do contest within that duration. Standard contest have configuration of contest scoreboard freeze time. Contest scoreboard that are displayed to contestant will be freezed after the freeze time.

• Virtual

  Virtual contest consist of two kinds of duration. The first duration is the contest total duration where contest can be entered within the duration. The second one is the contest virtual duration. Once a user has entered the contest, the total time of contest is using the contest virtual duration.

  Virtual contest can be started by contestant or team coach based on the configuration.

Scope¶

Currently, there are two contest scopes:

• Public

  In public contest, any user can register to the contest. Public contest is also known as open contest. Public contest have configuration of register time (user can only register within this time) and max registrants (the total number of maximum user registered).

• Private

  In private contest, users are registered manually by contest supervisors or managers. Only registered user can enter the contest.

Style¶

Currently, there are two contest styles:

• IOI

  IOI style contest supports partial score between 0 and 100. In IOI style, the time contestant uses to solve the problem is not counted for ranking. That means there could be two or more contestants with the same rank.

• ICPC

  ICPC style contest doesn’t support partial score, it only has 0 or 1 score. In ICPC style, there is a concept of penalty where a wrong answer will get time penalty. Besides wrong answer, a submitted answer by default get penalty based on the passed time since contest start. ICPC contest have configuration of the time penalty of wrong submission.

Usage¶

Development¶

• A class is not found, but it exists, how?

  Try to clean the project and its dependencies.

• You encounter no value error after submitting form, what to do?

  Try to clean the project and its dependencies.

• I don’t want any red underscore in my IntelliJ IDEA?

  Ashar please answer this