Datasets

This page provides the dataset-related information of 3D gaze estimation.

In this page,

  • we introduce the data pre-processing of each datasets.
  • we provide the code of data pre-processing.
  • we summarize the links of datasets.

We invite you to use our data processing code, and add your result into the benchmark.

Note that, you should first download the gaze rectification code data_processing_core.py from this page before using our data processing code.

If you use the codes in this page, please cite our survey:

@article{Cheng2021Survey,
    title={Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark},
    author={Yihua Cheng and Haofei Wang and Yiwei Bao and Feng Lu},
    journal={arXiv preprint arXiv:2104.12668},
    year={2021}
}

MPIIGaze

This processed dataset contains eye images and gaze directions of each eyes.

Data pre-processing

We follow the evaluation protocol of MPIIGaze. They provide a evaluation set containing 15 subjects and 3,000 eye images of each subject. The 3,000 eye images are consisted of 1,500 left eye image and 1,500 right eye image. We normalize the evaluation set with the same method as [1]. The reference point in the normalization is set as eye centers. We also flip the right eye image as [1]. After the normalization, eye images are histogram-equalised to handle various illumination. In our benchmark, we use leave-one-person-out strategy for evaluation.

Codes of pre-processing

Please download here for the source code. The code contains following parameters.

root = "/home/cyh/dataset/Original/MPIIGaze"
sample_root = "/home/cyh/dataset/Original/MPIIGaze/Origin/Evaluation Subset/sample list for eye image"
out_root = "/home/cyh/dataset/EyeBased/MPIIGaze"
  • The root is the path of MPIIGaze.
  • The sample_root indicates the sample list in MPIIGaze.
  • The out_root is the path for saving result.

To use the code, you should first set the three parameters. Then, run

python data_processing_mpii.py

The normalized data is saved in the out_root. We also provide the guide for using our normalized data.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@inproceedings{zhang15_cvpr,
    Author = {Xucong Zhang and Yusuke Sugano and Mario Fritz and Bulling, Andreas},
    Title = {Appearance-based Gaze Estimation in the Wild},
    Booktitle = {Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    Year = {2015},
    Month = {June}
    Pages = {4511-4520} 
}

MPIIFaceGaze

This processed dataset contains face and eye images. It also provides gaze directions of each face.

Data pre-processing

We follow [2] to process dataset. This dataset is an extensive version of MPIIGaze. We also use the evaluation dataset of MPIIGaze for evaluation. Detailly, for each eye image in the MPIIGaze, we get the corresponding face images. The normalization is done in the face image. Note that, the face image is flipped when the original sampled eye image is a right eye image as [2]. After normalizing the face image, we crop eye images from the face image. In our benchmark, we use leave-one-person-out strategy for the evaluation.

Codes of pre-processing

Please download here. The code contains following parameters.

root = "/home/cyh/dataset/Original/MPIIFaceGaze"
sample_root = "/home/cyh/dataset/Original/MPIIGaze/Origin/Evaluation Subset/sample list for eye image"
out_root = "/home/cyh/dataset/EyeBased/MPIIGaze"
  • The root is the path of MPIIFaceGaze.
  • The sample_root indicates the sample list in MPIIGaze. Note that, this file is not contained in MPIIFaceGaze. You should download MPIIGaze for this file.
  • The out_root is the path for saving result.

To use the code, you should set the three parameters first., and run

python data_processing_mpii.py

The normalized data is saved in out_root. We also provide the guide for using our normalized data.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@inproceedings{zhang2017s,
    title={It’s written all over your face: Full-face appearance-based gaze estimation},
    author={Zhang, Xucong and Sugano, Yusuke and Fritz, Mario and Bulling, Andreas},
    booktitle={Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference on},
    pages={2299--2308},
    year={2017},
    organization={IEEE}
}

EyeDiap (eye)

This processed dataset provides eye images and gaze directions of each eye.

Data pre-processing

We select VGA videos of screen targets and sample one image per fifteen frames. We then truncate the sampled image of each video to ensure the same number of images. We use the method proposed in [3] for normalization. Note that, only left eye images are used for evaluation. The reference point is set as eye center. After the normalization, eye images are histogram-equalised to handle various illumination. In our benchmark, we randomly divide whole subjects into four clusters and perform four-fold cross validation.

Codes of pre-processing

Please download from here. The code contains following parameters.

root = "/home/cyh/dataset/Original/EyeDiap/Data"
out_root = "/home/cyh/dataset/EyeBased/EyeDiap"
  • The root is the path of original EyeDiap dataset.
  • The out_root is the path for saving result file.

To use the code, you should first set the two paramters, and run

python data_processing_diap.py

The normalized data is saved in out_root. We also provide the guide for using our normalized data.

please download here for cluster_diap.py.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@inproceedings{eyediap,
    author = {Funes Mora, Kenneth Alberto and Monay, Florent and Odobez, Jean-Marc},
    title = {EYEDIAP: A Database for the Development and Evaluation of Gaze Estimation Algorithms from RGB and RGB-D Cameras},
    year = {2014},
    isbn = {9781450327510},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/2578153.2578190},
    doi = {10.1145/2578153.2578190},
    booktitle = {Proceedings of the Symposium on Eye Tracking Research and Applications},
    pages = {255–258},
    numpages = {4},
    keywords = {natural-light, database, RGB-D, RGB, remote sensing, gaze estimation, depth, head pose},
    location = {Safety Harbor, Florida},
    series = {ETRA '14}
}

EyeDiap (face)

This processed dataset contains face and eye images. It also provides gaze directions of each face.

Data pre-processing

We select VGA videos of screen targets and sample one image per fifteen frames. We then truncate the sampled image of each video to ensure the same number of images. We use the method proposed in [3] for normalization. The reference point is set as the center of two eyes. We crop the eye image from the normalized face image. Eye images are also histogram-equalised to handle various illumination. In our benchmark, we randomly divide whole subjects into four clusters and perform leave-one-out evaluation strategy in the four clusters. Note that, the clusters are same as the clusters in EyeDiap (eye center).

Codes of pre-processing

Please download here. The code contains following parameters.

root = "/home/cyh/dataset/Original/EyeDiap/Data"
out_root = "/home/cyh/dataset/FaceBased/EyeDiap"
  • The root is the path of EyeDiap.
  • The out_root is the path for saving result.

To use the code, you should first set the two paramters, and run

python data_processing_diap.py

The normalized data is saved in out_root. We also provide the guide for using our normalized data.

please download here for cluster_diap.py.

The original dataset can be downloaded from here.

If you use this dataset, please cite:

@inproceedings{eyediap,
    author = {Funes Mora, Kenneth Alberto and Monay, Florent and Odobez, Jean-Marc},
    title = {EYEDIAP: A Database for the Development and Evaluation of Gaze Estimation Algorithms from RGB and RGB-D Cameras},
    year = {2014},
    isbn = {9781450327510},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/2578153.2578190},
    doi = {10.1145/2578153.2578190},
    booktitle = {Proceedings of the Symposium on Eye Tracking Research and Applications},
    pages = {255–258},
    numpages = {4},
    keywords = {natural-light, database, RGB-D, RGB, remote sensing, gaze estimation, depth, head pose},
    location = {Safety Harbor, Florida},
    series = {ETRA '14}
}

UT-Multiview

Data pre-processing

UT-Multiview dataset provides processed data. We divide the whole subjects into three clusters and each cluster contains 16 subjects. In our benchmark, we perform three-fold cross validation. Following [4], we use synthesized set for training. We use the collected data for test.

Codes of pre-processing

Please download here. The code contains following parameters.

root = "/home/cyh/dataset/Original/UT/Origin"
out_root = "/home/cyh/dataset/EyeBased/UT"
  • The root is the path of original UT dataset.
  • The out_root is the path for saving result.

To use the code, you should first set the two paramters, and run

python data_processing_ut.py

The normalized data is saved in out_root. We also provide the guide for using our normalized data.

please download here for cluster_train_ut.py. and here for cluster_test_ut.py.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@InProceedings{Sugano_2014_CVPR,
    author = {Sugano, Yusuke and Matsushita, Yasuyuki and Sato, Yoichi},
    title = {Learning-by-Synthesis for Appearance-based 3D Gaze Estimation},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2014}
}

ETH-XGaze

Data pre-processing

ETH-XGaze dataset provides processed data. The dataset provide training and test set. We directly follow the protocol of this dataset.

Codes of pre-processing

Please download here. This code only convert the original data into a standard input file for our code.

The code contains following parameters.

path = "/home/cyh/dataset/Original/ETH-Gaze/test"
imo_path = "/home/cyh/dataset/FaceBased/ETH-Gaze/Image/test"
annoo_path = "/home/cyh/dataset/FaceBased/ETH-Gaze/Label/test.label"
test = True
  • The path is the path of original ETH-Gaze dataset. It indicate the converted folder. It can be ETH-Gaze/train or ETH-Gaze/test.
  • The imo_path is the path for saving images.
  • The annoo_path is the path for saving label file.
  • The test is True when the path is ETH-Gaze/test and False when the path is ETH-Gaze/train.

To use the code, you should first set the four paramters liking

path = "/home/cyh/dataset/Original/ETH-Gaze/test"
imo_path = "/home/cyh/dataset/FaceBased/ETH-Gaze/Image/test"
annoo_path = "/home/cyh/dataset/FaceBased/ETH-Gaze/Label/test.label"
test = True

and run

python data_processing_eth.py

You also need to modify the paramters liking:

path = "/home/cyh/dataset/Original/ETH-Gaze/train"
imo_path = "/home/cyh/dataset/FaceBased/ETH-Gaze/Image/train"
annoo_path = "/home/cyh/dataset/FaceBased/ETH-Gaze/Label/train.label"
test = False

and run again

python data_processing_eth.py

Note that, in our benchmark, we split the training set into a new training set and validation set. The validation set contains the data of subject 27, 43, 51, 60, 69.

We also provide the guide for using our normalized data.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@inproceedings{Zhang_2020_ECCV,
    author    = {Xucong Zhang and Seonwook Park and Thabo Beeler and Derek Bradley and Siyu Tang and Otmar Hilliges},
    title     = {ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation},
    year      = {2020},
    booktitle = {The European Conference on Computer Vision (ECCV)}
}

Gaze360

Data pre-processing

We directly use the provided data of Gaze360. The dataset already splits the whole dataset into training, test and valiation set. Note that, some images in Gaze360 only capture the back side of the subject. These images are not suitable for appearance-based methods. Therefore, we first clean the dataset with a simply rule. We remove the images without face detection results based on the provided face detection annotation.

Codes of pre-processing

Please download here.

The code contains following parameters.

root = "/home/cyh/dataset/Original/Gaze360/"
out_root = "/home/cyh/dataset/FaceBased/Gaze360"
  • The root is the path of original Gaze360 dataset.
  • The out_root is the path for saving result file.

To use the code, you should first set the two paramters, and run

python data_processing_gaze360.py

We also provide the guide for using our normalized data.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@InProceedings{Kellnhofer_2019_ICCV,
    author = {Kellnhofer, Petr and Recasens, Adria and Stent, Simon and Matusik, Wojciech and Torralba, Antonio},
    title = {Gaze360: Physically Unconstrained Gaze Estimation in the Wild},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    month = {October},
    year = {2019}
}

Rt-Gene

Data pre-processing

We follow the evaluation protocol. The whole dataset is splited into three clusters. A three-folder cross validation should be done in the dataset. Note that, the dataset contains synthesis images and collected images. We use both two kinds of images for training and test in collected images.

Codes of pre-processing

Please download here and here.

The codes contain following parameters.

root = "/home/cyh/dataset/Original/Rt-Gene"
out_root = "/home/cyh/dataset/FaceBased/Rt-Gene"
  • The root is the path of original Rt-Gene dataset.
  • The out_root is the path for saving result file.

To use the codes, you should first set the two paramters, and run

python data_processing_rt_train.py
python data_processing_rt_test.py

We also provide the guide for using our normalized data.

The original dataset can be downloaded from here.

To apply our code on this original dataset, please also cite:

@InProceedings{Fischer_2018_ECCV,
    author = {Fischer, Tobias and Jin Chang, Hyung and Demiris, Yiannis},
    title = {RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments},
    booktitle = {The European Conference on Computer Vision (ECCV)},
    month = {September},
    year = {2018}
}

Reference

  1. Appearance-based Gaze Estimation in the Wild.
  2. It’s written all over your face: Full-face appearance-based gaze estimation
  3. Revisiting Data Normalization for Appearance-Based Gaze Estimation
  4. Learning-by-Synthesis for Appearance-based 3D Gaze Estimation