Some questions about the setting of the last experiment

[LenarBe]

Hello Qing145! I'm trying to reproduce the experiments of your great paper! But I got confused about the setting of the last experiment(to score the KG KnowLife). Did you put the Sigmoid func on the loss to get the scores? And did you use all the unlabelled KnowLife data as the training set to classfy the labelled 5000 triples? Sorry to bother you but it seems that the git don't have code about this part. Thank you for your answer!

[Qing145]

Dear LenarBe, thanks a lot for your question. To get the confidence score, we do use the Sigmoid func to normalize the confidence loss as defined in eq 11. Besides that, we train our model with the extracted subset from KnowLife, and manually labelled them for evaluation and visualization. We'll release the labelled dataset soon in our new benchmark paper.

[LenarBe]

Thanks a lot for your answer! But is it ok to use KnowLife as a benchmark now? I searched for it but the portal website seems to have missed something. Besides that, I see in the paper of KnowLife that the authors once labelled some triplets in the KG, and the labels seem to be still accessible now(But as I have no idea about using the original KG, I don't know whether it is trustworthy). Have you tried that part of data?

[Qing145]

The official website of KnowLife is down, but you can get the entire dataset by email to Dr. Ernst. And the labelled triples are valuable but not useful for building a subgraph for error detection since these labelled errors are distributed discretely throughout the entire KG. I mean the extracted subset should be a complete graph rather than a set of independent triples, since the global graph structure is indispensable for detection. Therefore, I strongly recommend using graph split methods to extract a subset and label each triple manually. Alternatively, conducting error detection on the entire KnowLife dataset may be an option, but it may not be feasible due to its large scale. I hope this information is helpful to you.

[LenarBe]

Oh, I got it.Thanks a lot for answering my question! Hope that everything goes well and looking forward to your new benchmark paper! (I have no question now and you can close this issue if there is no other problem,thanks again!)

[LenarBe]

Oh, I'm so sorry to bother you again. But I came across another problem in my attempt. I manully construct a KG with some errors and put it into this code, but it don't work very well. I read the codes again and think that, in the 'dataset.py' the t2h and h2t is constructed before injecting the anomalies and will be free from errors. I wonder if it is consistent with the predition of the method.Thanks again for your patient answering!