WORKSHOP SUB-TOPIC: Designing a Kaggle competition to do a POC on preliminary datasets of markes, initials & signatures

[Addaci]

@gbohner has suggested that the Signs of Literacy community set up a Kaggle research competition to perform a Proof of Concept on preliminary datasets of markes, initials & signatures. Gergo states that "The Kaggle community is adept at applying a range of potential algorithms quickly to solve similar problems, and it will give you a good idea of 1) how well AI can replace the volunteers on your task(s), and a starting point for 2) what are the kind of algorithms you might want to improve upon in a collaboration with a research lab."

[Addaci]

Kaggle competitions are based on supervised learning, and require both a training set ofdata for which the target predicted value already has a ground truth outcome available. Kaggle will use this ground truth as an answer key to score the accuracy of participants’ submissions in real time over the course of the competition. For more information see https://www.kaggle.com/host/business and https://www.kaggle.com/host/research.

Background

Some of the best work done on Kaggle has been on research competitions. They present new, unique machine learning datasets and use-cases to the community, and we want to make them available to Kagglers on a more regular basis.

Historically, the process of making research competitions available has been ad-hoc and highly circumstantial. Going forward, we want to make it easier for motivated research, academic, and non-profit groups to host open source challenges on Kaggle.

Opportunity

Kaggle will partner with organizations to host up to 5 pro-bono research competitions a year. We are asking interested organizations to submit a brief proposal for Kaggle’s consideration.

Requirements

To be eligible to host a research competition, the host organization must:

• Be an academic, research, or non-profit organization
• Have legal authorization to release the competition dataset publicly
• Post a cash prize
• Offer the prize on condition of winning solutions being open sourced

Selection Criteria

Kaggle will select research hosts on a rolling basis over the entire year. We will opportunistically reach out to prospective hosts based on the content of their application forms. Primarily, we will be looking for:

Fit - Is the project a cleanly structured supervised learning problem? Is there ample data available to support a successful competition?

Feasibility - Will it be logistically possible to make the data available for the competition? Is the hosting organization in a position to facilitate a successful competition?

Impact - What is the scope of the problem this competition addresses? How impactful would the open source solutions be on the industry this competition supports?

Kaggle and the prospective host will go through a vetting process before setting up and launching a competition. This vetting process will include a review of the dataset, some premodeling of the problem, and selection of an evaluation metric. The host organization will also need to sign a data-use agreement and be ready to transfer Kaggle the full value of the prize pool upon launch of the competition. Kaggle will facilitate the payment of prizes to winners at the competition close.

[Addaci]

We have posted a request for help on the Kaggle Twitter account.

[Addaci]

We submitted on Monday, April 31st 2018 a draft proposal to Kaggle for a Kaggle hosted pro bono funded Kaggle Research Competition. As background, Kaggle has access to a community of ca. 500,000 AI/machine learners/data scientists. It was set up as a crowdsourced approach to R&D in machine learning, and was acquired in March 2017 by Google. It is being maintained as a separate branded service within Google Cloud.

We have also publicised the proposed Kaggle Research Competition idea on LinkedIn and Twitter and it is attracting interest

https://www.linkedin.com/pulse/proposed-signs-literacy-kaggle-research-competition-2018-greenstreet/

We have received a quick and positive response from Maggie Demkin of Kaggle:

"Hello Colin,

Thank you for reaching out to Kaggle, this sounds like a very interesting opportunity. It makes sense to set up time to review your idea in more detail. During this meeting I can tell you more about Kaggle and what we do. I am located in California (Pacific Time), what are some dates/time in the next few weeks that work for you?

Thanks,
Maggie"

[Addaci]

@mhailwood @voetnoot @Giovanni1085 @gbohner @MvanErp Positive initial response from Dr Peter Bloomfield, AI/ML Engagement Manager, at Digital Capital in London, to my approach seeking to interest him and Machine Learners at the Digital Catapult in our proposed Kaggle competition:

"Hi Colin,

It was nice to meet you briefly, thanks for getting in touch! I'll have a think about the Kaggle competition and discuss it with some more of my team and get back to you! initially it sounds like good fun, so it would mostly be a case of being able to allocate resources to it

All the best

Peter
Peter Bloomfield PhD
AI/ML Policy + Engagement Manager
Check out the Machine Intelligence Garage! www.migarage.ai

Digital Catapult
101 Euston Road
London, NW1 2RA"

[Addaci]

Video call arranged with Maggie Demkin of Kaggle, Tuesday May 8, 2018 5pm – 5:45pm London "to provide you with a 20-30 minute overview on Kaggle and to answer any questions you have about hosting a competition". Also invited to call if available and interested @mhailwood @voetnoot @gbohner