Standard metrics for biometric system evaluation

This algorithm is a legacy one. The API has changed since its implementation. New versions and forks will need to be updated.
This algorithm is an analyzer. It can only be used on analysis blocks.

Algorithms have at least one input and one output. All algorithm endpoints are organized in groups. Groups are used by the platform to indicate which inputs and outputs are synchronized together. The first group is automatically synchronized with the channel defined by the block in which the algorithm is deployed.

Group: test

Endpoint Name Data Format Nature
scores_test tutorial/probe_scores/1 Input

Group: dev

Endpoint Name Data Format Nature
scores_dev tutorial/probe_scores/1 Input

Analyzers may produce any number of results. Once experiments using this analyzer are done, you may display the results or filter experiments using criteria based on them.

Name Type
hter float32
far_test float32
far_dev float32
frr_test float32
number_of_positives_dev int32
number_of_negatives_test int32
threshold float32
number_of_negatives_dev int32
frr_dev float32
roc_test plot/isoroc/1
number_of_positives_test int32
eer float32
roc_dev plot/isoroc/1

The code for this algorithm in Python
The ruler at 80 columns indicate suggested POSIX line breaks (for readability).
The editor will automatically enlarge to accomodate the entirety of your input
Use keyboard shortcuts for search/replace and faster editing. For example, use Ctrl-F (PC) or Cmd-F (Mac) to search through this box

An algorithm that implements standard metrics for biometric system evaluation.

Specifically, it returns:

  • eer: the equal error rate (EER) on the development set.
  • hter: the half total error rate (HTER) on the test set
  • far_dev: the false acceptance rate (FAR) on the development set
  • frr_dev: the false rejection rate (FRR) on the development set
  • far_dev: the false acceptance rate (FAR) on the test set
  • frr_test: the false rejection rate (FRR) on the test set
  • number_of_positives_dev: the number of positive (genuine) trials on the development set
  • number_of_negatives_dev: the number of negative (impostor) trials on the development set
  • number_of_positives_test: the number of positive (genuine) trials on the test set
  • number_of_negatives_test: the number of negative (impostor) trials on the test set
  • threshold: the threshold at the equal error rate on the development set
  • roc_dev: the receiver operating characteristic (ROC) curve on the development set according to the biometrics standard ISO/IEC 19795-1:2006(E)
  • roc_test: the receiver operating characteristic (ROC) curve on the test set according to the biometrics standard ISO/IEC 19795-1:2006(E)

This implementation relies on the 'measure' package from the Bob library.

Experiments

Updated Name Databases/Protocols Analyzers
smarcel/chichan/full_pre_mlbphs_projection/2/mobio-f_TT_MLBPH_PCA98_LDA300_postperf-iso mobio/2@female tutorial/eerhter_postperf_iso/1
smarcel/chichan/full_pre_mlbphs_projection/2/mobio-m_TT_MLBPH_PCA98_LDA300_postperf-iso mobio/2@male tutorial/eerhter_postperf_iso/1
smarcel/chichan/full_pre_mlbphs_projection/2/mobio-m_TT_MLBPH_PCA98_postperf-iso mobio/2@male tutorial/eerhter_postperf_iso/1
smarcel/chichan/full_pre_mlbphs_projection/2/mobio-f_TT_MLBPH_PCA98_postperf-iso mobio/2@female tutorial/eerhter_postperf_iso/1
smarcel/tpereira/full_isv_multi/2/btas2015_face-periocular_mobio-female_det mobio/1@female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_periocular_mobio-female_det_bobv2-0 mobio/1@female tutorial/eerhter_postperf_iso/1
tpereira/tutorial/full_lbphs/2/btas2015_LBPBaseline_face_cpqd-smartphone-male_det cpqd/1@smartphone_male tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv_multi/2/btas2015_face-periocular_cpqd-smartphone-male_det cpqd/1@smartphone_male tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_periocular_cpqd-smartphone-male_det cpqd/1@smartphone_male tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_face_cpqd-smartphone-male_det cpqd/1@smartphone_male tutorial/eerhter_postperf_iso/1
tpereira/tutorial/full_lbphs/2/btas2015_LBP-baseline_periocular_mobio-male_det mobio/1@male tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv_multi/2/btas2015_face-periocular_mobio-male_det mobio/1@male tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_periocular_mobio-male_det mobio/1@male tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_face_mobio-male_det mobio/1@male tutorial/eerhter_postperf_iso/1
tpereira/tutorial/full_lbphs/2/btas2015_LBPBaseline_face_cpqd-smartphone-female_det cpqd/1@smartphone_female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv_multi/2/btas2015_face-periocular_cpqd-smartphone-female_det cpqd/1@smartphone_female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_periocular_cpqd-smartphone-female_det cpqd/1@smartphone_female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_face_cpqd-smartphone-female_det cpqd/1@smartphone_female tutorial/eerhter_postperf_iso/1
tpereira/tutorial/full_lbphs/2/btas2015_LBP-baseline_periocular_mobio-female_det mobio/1@female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv_multi/2/btas2015_face-periocular_mobio-female_det mobio/1@female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_periocular_mobio-female_det mobio/1@female tutorial/eerhter_postperf_iso/1
tpereira/tpereira/full_isv/2/btas2015_face_mobio-female_det mobio/1@female tutorial/eerhter_postperf_iso/1
smarcel/tutorial/full_isv/2/mobio_male-gmm_100Gx10I-isv_50Ux10Ix4R-dct_12Bx8Ox45C-seed101 mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_ubmgmm/2/mobioMale_gmm_DCT12x8_100G mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_lbphs/2/mobioMale_lbphs12x8 mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_lbphs/2/atnt_lbphs12x8 atnt/1@idiap_test_eyepos tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_ubmgmm/2/mobioMale_ubmgmm_DCT12x8_100G mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_fisherface/1/mobioMale_fisherfaces_50and20comp mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_eigenface/1/mobioMale_eigenfaces_50comp mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_ubmgmm/2/bancaP_gmm_DCT12x8_100G banca/1@P tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_isv/2/bancaMc_isv_DCT12x8_100G_U50 banca/1@Mc tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_isv/2/xm2vtsLp1_isv_DCT12x8_100G_U50 xm2vts/1@lp1 tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_isv/2/mobioMale_isv_DCT12x8_100G_U50 mobio/1@male tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_isv/2/bancaP_isv_DCT12x8_100G_U50 banca/1@P tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_isv/2/atnt_isv_DCT12x8_100G_U50 atnt/1@idiap_test_eyepos tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_fisherface/1/fullfisherfaces_10and4ncomp atnt/1@idiap_test_eyepos tutorial/eerhter_postperf_iso/1
tutorial/tutorial/full_fisherface/1/fullfisherfaces_5and2ncomp atnt/1@idiap_test_eyepos tutorial/eerhter_postperf_iso/1

This table shows the number of times this algorithm has been successfully run using the given environment. Note this does not provide sufficient information to evaluate if the algorithm will run when submitted to different conditions.

Terms of Service | Contact Information | BEAT platform version 1.3.1rc1 | © Idiap Research Institute - 2013-2019