Home › Deep tech

Deep tech · hard problems, measured

Deep tech

Hard research problems where one small group can still produce a result that counts. The common thread: the quality of the data decides the quality of the model, and that can be measured instead of argued about.

The projects

Start with the list of hard problems, then pick a project that tests one of them.

The hardest unsolved problems

79 problems in 10 groups

From fusion to forgetting: what each problem is, why it's hard, and which ones one person or a small team can actually test.

Hinglish pre-training data

an open corpus for how India types

Hundreds of millions of people type Hindi in English letters, yet the biggest open web corpus holds only about 0.2 GB of it, and a third is junk. Build a clean, open one and prove it helps.

A data benchmark for robot learning

same robot, same training, different data

Many methods claim to pick the best robot demonstrations, but each is tested on its own setup. One fixed test bench shows which ones help once the robot acts.

How these projects are run

Each one fixes its success test before the work starts, uses free and open tools wherever they exist, keeps cloud GPU spend small (hundreds of dollars, not millions), and publishes data, code and results in the open, including results that show no effect.

Want to help? These need people who like careful experiments: data engineers, ML engineers, researchers and students. Write to basant@cleandataengineering.ai and say what you would like to pick up.