Train set/test set: We would like to point out that most prior work in unsupervised classification use both the train and test set during training. We believe this is bad practice and therefore propose to only train on the train set. The final numbers should be reported on the test set (see table 3 of our paper). This also allows us to directly compare with supervised and semi-supervised methods in the literature. We encourage future work to do the same. We observe around 2% improvement over the reported numbers when including the test set. Reproducibility: We noticed that prior work is very initialization sensitive. So, we don't think reporting a single number is therefore fair. We report our results as the mean and standard deviation over 10 runs. Please follow the instructions underneath to perform semantic clustering with SCAN.,这段话是什么意思
时间: 2024-04-29 21:19:44 浏览: 113
Effects of examiners' prior knowledge of subjects' ethnicity and intelligence on the scoring of responses to the Stanford-Binet scale
这段话是在介绍一个名为SCAN的无监督分类方法的实验设置和结果报告。其中,作者指出之前的一些无监督分类方法在训练时同时使用了训练集和测试集,这种做法是不好的,因为会导致模型泛化能力的过度估计。因此,作者建议只在训练集上进行训练,并在测试集上报告最终结果,以便与其他监督和半监督方法进行比较。作者还观察到,之前的一些方法非常依赖于初始化,因此为了保证结果的可重复性,作者进行了10次实验,并报告了平均值和标准差。最后,作者提供了使用SCAN进行语义聚类的指南。
阅读全文