Reviewer has chosen to be Anonymous

Overall Impression: Excellent

Content:
Technical Quality of the paper: Excellent
Originality of the paper: Yes
Adequacy of the bibliography: Yes

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Good

Detailed Comments:

The paper is an extended version of previous work by the authors [ref35], which proposed and evaluated modifications to a popular embedding technique for EL++ ontologies that aim to leverage the ontology's deductive closure to ensure that negative sampling distinguishes between statements that are unprovable and provably false and designs specific losses for different negative axiom types.
The extension delves deeper into deductive closure and provides additional experiments with different ontologies and tasks.

All code and data are available in an open repository.

The methodology is interesting and sound, and the results support the discussion. The paper’s structure could be improved for clarity.

Specific comments:

[1] The organization of the Methods and Results section is not clear. The Results section describes the novel methodological contributions and experimental results. While this type of organization may be common in other areas (e.g. biology) I don't think it works well here. I suggest the authors reorganize these sections to clearly present their methodological contributions, experimental design and evaluation results.

[2] The authors can be more positive about their contributions. It is presented here in very neutral terms. For instance, I strongly suggest that the authors rewrite section 5.1 in a way that more clearly demonstrates the novelty and potential impact.

[3] In tables 1,2 and 3, bold and underlined are confusing. Do you mean best results for each embedding method by filtered dataset? If so, you are missing several bolds and underlines.

[4] Can you explain why Table 2 presents only non-filtered metrics?

[5] Is Equation 6 this is the same as the "positive loss" for disjointness?

[6] Fig. 3. Is missing a caption, and 3b is hard to interpret due to the scale of the y axis.

[7] P3 L18-22: These two sentences are not clear. Grammar could be improved, as well as a more detailed explanation of the achievement of ref30.

[8] Please clarify that the extended version includes additional experiments with different ontologies and tasks.

Minor:

P3 L 23 missing space in: [39]or
P2 L34: The GCIs and RIs would be easier to read if presented in a table format.

Reviewer has chosen to be Anonymous

Overall Impression: Weak

Content:
Technical Quality of the paper: Average
Originality of the paper: No
Adequacy of the bibliography: Yes

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Good

Detailed Comments:

The authors describe a method to use the embeddings of the description logic EL++ for knowledge base completion. They designed negative losses for three embedding methods and evaluate this approach on a few ontologies. The results indicate that the proposed approach improves on the baseline embedding method.

Strengths

1. An extension that makes use of negative losses and deductive closure to improve ontology completion.
2. An approach that is independent of the underlying embedding technique.
3. Fairly well-written and explained. But some parts of the manuscript are not clear.

Weaknesses

1. The addition of work to [35] is not clear. I am also not sure if the additions are sufficient for a journal. These additions should be explained in more detail.
2. Although there is a mention of 3-4 ontologies for evaluation, in the results tables, evaluation on only one ontology is shown. They are not even present in the Appendix. Why is that the case?
3. The evaluation is performed only on one task (ontology completion) and subsumption prediction on only one ontology. How are the results of these two tasks on other ontologies?

Other questions/comments

1. The following paper is also relevant to this work. “EmELvar: A NeuroSymbolic Reasoner for the EL++ Description Logic. Biswesh Mohapatra, Sumit Bhatia, M. Raghava Mutharaju, and G. Srinivasaraghava. https://ceur-ws.org/Vol-3123/paper6.pdf.
2. What is meant by filtered metrics?
3. The negative losses in Section 5 is the main contribution and should be discussed earlier and not part of the Experiments section.
4. The purpose of Algorithm 2 is not clear. It needs to be better motivated and explained, perhaps with the help of an example.
5. In lines 47, and 48 of page 11, it is mentioned that filtered negatives provide a faithful representation of GCI3. Why is this the case?
6. The difference between Figures 2(a) and 2(b) is not clear.
7. What are non-filtered metrics and filtered metrics?
8. The improvements seem to be marginal, if at all. The use of negative losses does not seem to make too much of a difference.

Addressing all the points mentioned in the Weaknesses section may require major changes to the manuscript unless the authors have already performed these evaluations. Considering that, I suggest a major revision.

Reviewer has chosen to be Anonymous

Overall Impression: Weak

Content:
Technical Quality of the paper: Weak
Originality of the paper: Yes, but limited
Adequacy of the bibliography: Yes, but see detailed comments

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Limited
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Average

Detailed Comments:

In this work, the authors take a closer look at EL embedding methods as well as the format of axioms in ontologies used for benchmarking. There are no theoretical proofs, the authors focus on an evaluation of current methods from a practical point of view. The main idea of the paper is to improve the strategy of negative sampling so as to avoid using axioms that are entailed.

Detailed comments:

Clarify that such deductive closure is finite for normalized axioms. In the description logic literature, the EL++ ontology language is often defined without normalization (see the paper Pushing the EL envelope or Pushing the EL envelope further) and the deductive closure can be infinite. Although it is common to normalize ontologies, this may not necessarily be the case in practice. In fact many EL ontologies used in practice, such as SNOMED CT or Galen, are not normalized. E.g., there are axioms of the form A\sqsubseteq \exists r. (B \sqcap \exists r. C), even if they are fewer in quantity.
The papers on EL embeddings introduce new symbols and modify these original ontologies so as to end up with normalized ontologies for benchmarking. But since this is not how the language is used in practice, it would be better to differentiate this explicitly. One can introduce a name such as EL^n++ whenever one wants to refer to a normalized ontology language. If the authors would like to keep EL++ instead of giving a new name to the normalized language, then at least a comment about this fact should be added.

Here are some recent papers on EL embeddings that are not referred to but are related.

Hui Yang, Jiaoyan Chen, Uli Sattler: TransBox: EL++-closed Ontology Embedding. CoRR abs/2410.14571 (2024)
https://arxiv.org/abs/2410.14571

Victor Lacerda, Ana Ozaki, Ricardo Guimarães: Strong Faithfulness for ELH Ontology Embeddings. TGDK 2(3): 2:1-2:29 (2024)
https://drops.dagstuhl.de/storage/08tgdk/tgdk-vol002/tgdk-vol002-issue00...

Victor Lacerda, Ana Ozaki, Ricardo Guimarães: FaithEL: Strongly TBox Faithful Knowledge Base Embeddings for Eℒ. RuleML+RR 2024: 191-199
https://link.springer.com/chapter/10.1007/978-3-031-72407-7_14

Abstract - The authors claim that EL embedding methods do not use the deductive closure of an ontology to identify statements that are inferred but not asserted. Recent works are using the canonical model for EL ontologies (see last two references). The canonical model has not only what is explicit but also the deductive closure. Since these works are using the deductive closure, the abstract needs to be rephrased and we suggest the authors consider these works as well (no need to change the experiments, just mention that this strategy has been explored).

Section 2: Preliminaries
The last two lines on the semantics need to be fixed.

in the line that you have \exists r.A, there is the part {a \in \Delta^I … }. Note that a is an element of \bold{I}, not of \Delta^I, because you did not define \Delta^I as a set that contains \bold{I}.

In the last line, it should be {a}^I = {a^I} instead of {a}^I = {a}.

It would be better to describe the normalization step in detail. You can see an example here: https://link.springer.com/chapter/10.1007/978-3-642-33158-9_4 in Figure 3.

Section 2.2: “axioms to a knowledge base that are not explicitly represented” -> (suggestion, something like (?)) that hold but are not present (and not in the deductive closure).

Section 3.3
[23] explores -> Mention the name of the author(s). Same for other similar cases

Section 3.4 Notion of entailment: “when a model” -> whenever a model (because when a model is giving the idea that only one model is needed but in reality this is quantified over all models). Perhaps rewrite this whole part and give a more formal definition using interpretations? This would be better since you already defined interpretations before in the paper.

Section 4.1.2
.. to EL fragment… -> to *the* EL fragment

… ; to avoid… -> … . To avoid…

Page 6 line 3: … for Food Ontology… -> for the Food Ontology

Page 6 line 24 “or semantically valid concept” this sounds strange because semantics is related to meaning and usually this kind of replacement happens at a syntactic level, not taking into account the meaning of the concept that is used to replace other than it is not entailed. Or maybe I misunderstood, what do you mean here exactly?

Section 5.1.1. Please say what is what, to all the symbols that suddenly appear in equations 6, 7, 8 *before they appear* and also give intuition. You write a few things on page 7 but that is too late.

Section 5.2.1 You can define deductive closure earlier, in the preliminaries, since this is a notion from the literature, not a notion that you introduce (and also make that comment about it being finite for normalized axioms, so that you can compute it).

Algorithm 1, these rules are inference rules for EL that you mention that you are taking from ELK. Then, no need to write as an algorithm and it is strange that you put in an algorithm environment without making explicit what is the input and what is the output and whether these rules are complete (even though they are sound). Since these rules exist you could add them in the appendix and just that in the main text what you do with the axioms inferred by applying the rules.