Pre-evaluation and Filtering of Jury Membership Applications

Prelude
Every year, A’ Design Award & Competition receives hundreds and sometimes thousands of jury member applications. It is not always possible to go through each and every jury member application without help of an algorithm that provides a preliminary screening and filtering of applications. In order to make it possible to select the best jury members while reviewing all applications, we had to develop an algorithm that could help us prefilter the applications and present us with a limited set of jury membership applications that are already perfect. The jury member selection is based on the following algorithm. This algorithm is made available to all design award juror applicants as part of our transparency policy. In the end, the algorithm provides us with a score, we consider the score as a quality of the application itself; we reject applications with low scores, and review applications that satisfy a base level of minimum score requirement. This allows us to evaluate jury memberships in a reasonable manner. Maximum score the algorithm presents is 25+15+20+30+10=100. We automatically reject all entries that have less than 70 score.

Jury Member Type (Max.25)
First of all, the job definition is important for us, the A' Design Award jury is composed of five types of jurors; Academics, Press Members, Design Professionals, Focus Group and Entrepreneurs. To vote the entries the best way possible, we want to have a high amount of Academics. To disseminate the voted entries as well as to provide us with a media perspective, we also aim to have high amount of Press Members. To provide a creative look we seek votes from Design Professionals, and to provide us with market feedback we seek input from Entrepreneurs. The algorithm assigns following base scores to each jury member group: Academics (20), Press Members (25), Design Professionals (15), Entrepreneurs (10), and Focus Group (5). Focus group are design enthusiasts and general public, include but not limited to celebrities that like and understand design but are not designers themselves.

Age (Max.15)
While we of course believe that creativity and general intellect is not correlated to age; and perhaps it is reduced by age, i.e. children are more creative and smart than adults, we also understand and agree that wisdom on the other hand is usually positively correlated with age; most people get wiser with time, and experience is what counts. On the other hand, we want to have young opinions as well as professional ones, therefore our algorithm incorporates the following function to determine which score to provide to jurors based on their age; For this purpose we define maturity age as 45, which is a person with a lot of experience yet also with a still dynamic mind. We want to stay 15 years within the M, maturity age. But we also want to have older professionals as they add a level of prestige and professionalism to the competition, until a certain age, such as 75 where the gains from their age is offset by their performance of voting entries. In this case the algorithm is as follows: if (AGE<=45){S=max(AGE-(30),0);} if (AGE>45 && AGE<=50) {S= abs(AGE-60);} if (AGE>55 && AGE<=75){S=5;} if (AGE>75){S=0;} . This formula provides a maximum score of fifteen (15) at age of 45, and fourteen (14) points for people with ages 44 or 46, and provides zero (0) score to people younger than 30, and provides five (5) score  for people between 55 and 75 and provides zero (0) score to people over 75.

Jury Photograph (Max. 20)
It is very important for us to have the jury photograph in the correct size and dimensions. Basically, if a juror is not able to upload their photo in correct size, how can they vote the entries following our guidelines? We consider providing correct jury photograph as an indicator of following guidelines and methodology. We provide ten (20) score if photo is uploaded and is correct (correct photo is a square image with at least 900px width and height). We provide five (10) score if photo is high-res uploaded but not correct (should be a large image with at least 900px width). We provide five (5) score if low-res photo is uploaded but not correct. We provide zero (0) score if there is no photograph. Then, we manually review the quality of photograph itself;  this process is done after the automated score is calculated; for example if the juror photo is not a bust or portrait image as noted in our guidelines we reduce points. We also review if the photograph was taken professionally and reflects the juror in their best selves.

Categories Selected (Max.30)
For main category, juror is given eight (8) points and for each further selected category, the juror is given two (2) points. This is important because we want jurors to vote a wide range of categories, not just their specialization; this is done for purposes of dissemination and error reduction and to have outside perspective for various categories. Of course jurors should select adjacent categories; for example an architect should perhaps select Architecture as main category, then interior design, urban design, furniture design and other categories where their experience and knowledge is truly useful for determining which designs are good.

Biography (Max.10)
We want to ensure jurors have provided a correct, concise biography to be used together with their jury application. We want to avoid cv copy paste, too short or too long descriptions. The following algorithm is used for calculating the score for biography section where L is the Length of Jury Biography in terms of number of characters written. if (L>2500){S=2;} if (L>1500 && L<=2500){S=5;} If (L>500 && L<=1500){S=10;} If (L>200 && L=<500){S=5;} if (L>100 && L<=200){S=2;} if (L>50 && L<=100){S=1;} if (L<50){S=0;} If a biography is provided, and is between 500 and 1500 characters, juror is given a score of ten (10).  If the biography length is between 1500 and 2500, the score is five (5), the score is zero (0) when the length is less than 100 characters, one (1) if less than 200 characters, five (5) if between 200 and 500 characters. However the computer cannot calculate the quality of text itself, therefore we later also manually review and deduct points for the structure of the biography, if the biography does not make sense or if the language is not correct or if there are too many grammatical errors, we reduce the score significantly.

Manual Review Items for Reducing Automatically Calculated Score
Quality and conformity of Photograph with our guidelines (must be a professionally taken portrait image), Quality of Biography description (should be in good English, making good sense, there should not be email, phone number or website in biographies because we do not want participants to solicit votes from jurors, this is important, jury application bio should not be a letter of intent; it must solely be the biography). Truthfulness of information and relevance of categories to the profession indicated and jury type selected.

Final Selection
It is important to remind you that the algorithm is solely for rejecting and reducing the number of applications but not for accepting or approving applications; approvals and acceptance shall be made by reviewers and jury coordinators who decide and select jurors among already filtered applications. When selecting jurors, the following shall be considered in detail: CV of jurors especially; awards won, exhibitions organized or partaken, achievements and successes, publications, education and overall biography.