By Anonymous User
Review Details
Reviewer has chosen to be Anonymous
Overall Impression: Good
Content:
Technical Quality of the paper: Good
Originality of the paper: Yes
Adequacy of the bibliography: Yes
Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Average
Detailed Comments:
Evaluation criteria for Survey (NeuroSymbolic Journal):
(1) Suitability as introductory text, targeted at researchers, PhD students, or practitioners, to get started on the covered topic.
Yes, definitely. However the paper needs to be improved.
(2) How comprehensive and how balanced is the presentation and coverage; in particular the survey should not be a platform to promote the authors' work.
This is quite balanced with a focus on papers in 2020-2025.
(3) Readability and clarity of the presentation.
The section nummering is downright bad: in the text references to section X, but the sections are not numbered at all.
The readability and clarity can be improved. Below are a number of suggestions to improve the paper.
(4) Importance of the covered material to the broader Neurosymbolic AI community.
Their analysis in themes (performance, understandability, reliability and ethics), function roles (=perception, knowledge, reasoning, planning/control and oversight) and interaction patterns is interesting for the community. I’m not able to check whether the categorization of all papers is correct, but it seems to me an in-depth study.
=======================================================================
This survey is based on papers from 2020-2025. They discuss neurosymbolic AI in four themes: performance, understandability, reliability and ethics. They describe the interface patterns, taking into account the input/output of neural components, or the use of constraints by the neural components. Furthermore they use five function roles in an AI system: perception, knowledge, reasoning, planning/control and oversight.
This survey is about very specific notion of Neurosymbolic AI, namely the method has at least a neural network component.
Title: A Survey of Neurosymbolic AI: foundations, advance, and future trajectories.
After reading I asked myself whether it is clear from the paper:
What is considered as the “foundations” “advances” “future trajectories”
I think the authors can emphasize this a bit more in the conclusion.
The authors mentioned the following contributions:
(i) Foundational anchors: Q: (Q: these are the 4 themes, the function roles, and the interaction patterns?)
(ii)Mapping methods x system functions (+benchmarks and evaluation measures): Q: thus evaluation of the functions (=perception, knowledge, reasoning, planning/control and oversight.)
(iii) evaluate each theme (evaluation measures, benchmarks, reproducibility signals). (Which table??)
(iv) theme analysis of system-design pitfalls (eg. Cost vs. garantees)
Do the authors mean trade-offs instead of pitfalls? (Described via interface patterns)
(v) future directions / challenges (concrete evaluation criteria and test considerations)
Do the authors mean evolution criteria for functions or themes?
My suggestion is to refer to the tables (sections) that belong to the above contributions.
Where do those five function roles in an AI system (perception, knowledge, reasoning, planning/control and oversight) come from?
Two comments on those five chosen function roles: (1) Knowledge seems a strange “type” compared with the other roles, and (2) “explaining” would that not also be a function rol?
How do the authors consider the “structures traces” as a reasoning operator?
Section Background:
(In the classical sense….). Decide whether it is important what is written in brackets. If it is important then remove the brackets (include it in the main text, and make two sentences), or delete (…).
Problem Statement:
Make more explicit that there is a need for benchmarking measures.
Page 6: “AI subdomains” Do the authors mean “the function roles in AI systems” or do they mean a domain like medical or subdomains like NLP, vision.
Page 6: What is the difference between this survey and the surveys from (Bubeck 2023, Rezazadegan 2024)? Do they have a different focus, what are the main takeways from those surveys? (Later I saw the table 11). Might be good to add a sentence here (page 6) about the main difference.
Page 6: What are end-to-end implications?
“A consistent mapping to system functions and evaluation measures”. Those evaluation measures are for the four themes or for the system functions? I would expect for the four themes.
Comments wrt. Table 2:
Page 7: confusing remarks about table 2:
[1] (ii) table 2 is a mapping from methods to system functions (with benchmarks, evaluation measures)
[2] In the text: “The summary matrix of Table 2 provides a consistent thread for mapping advances to goals, placing results within a practical system setting, and clarifying where knowledge and explanations originate.”
[3] Caption table 2: survey theme, system function, interface pattern, evaluation levers.
Where are the benchmarks from [1] in the table?
Wrt [2]. The goals are the functions or the themes? What about the Knowledge and Explanations?
Which interface patterns do you consider? This is not clear.
The “typical methods (example)” are the interface patterns? (It seems they are not always match with the interaction patterns in table 4).
Page 8: Overview of the paper.
Section numbers do not make any sense.
Page 9: Source selection. The authors submit a survey to the Neurosymbolic Journal, but they do not mention the journal as “journals relevant to Neurosymbolic AI and KR”.
Appendix:
Can you motivate why the queries in the table are dependent of the source?
Page 9:
Items were tagged with primary and optional with secondary tag.
Which “tags” have been used. Are they based on the papers, or did you start with a set of tags and then adapted the set based on the papers?
Critical analysis:
(i)Problem abstractions and integration patterns
(ii) evaluation designs, datasets, evaluation measures
(iii) limitations and threats to validity
Q: Do the authors mean with the problem abstractions and the system functions the same?
Then they summarize again what to expect:
System functions - evaluation per theme - reproducibility (including ablations for robustness) - discussion&future work.
New is the reproducibility aspect?
Earlier (overview of the paper page 8), they mentioned:
Theme organization:
Problem framing, representative advances, Evaluation/benchmarks, Limitations, Takeaway.
Table 1 gives the possible interface pattern of the neurosymbolic systems?
I think “neuro —> symbolic” is an interface pattern. However the “cost profile” does not seem to me an interface pattern.
What are the dimensions in table 1? Or are those the tags?
Page 11,12:
A number of citation roles are given: Spine, pattern exemplar, evidence citation, context/background,/postion/opinion.
Q1: where are those citation roles used? Table 1 is of the type “Spine”? If so, add this in the caption of table 1.
Page 12: Last sentence: “This table illustrates the protocol in Table 3” Which table?
Page 13: Can the authors clarify “it does not imply that all listed systems are directly comparable or that any dimension transfers across tasks”.
Note on page 13:
- “this table” —> table 4.
- “Problem setting” = functional role.
Table 4: U expected here the function roles (=perception, knowledge, reasoning, planning/control and oversight.)
Where do those “problem settings” come from?
I would like to suggest to explain the relation between table 1, table 2 and table 4 more explicite.
page 14: use the same name of the themes, so first section is “Performance”. (Performant AI: …)
Page 14:
Refer in the text to Table 5.
Page 21: “we review works that make AI reasoning more interpretable”. Do the authors mean “AI” or “Neuro”?
Page 27: “In the running example, …” Eh which running example?
Page 29: “In the running example, …” Which running example?
Page 30-34: this part needs to be improved.
Page 30-34: What is the story line of this section, and how fits this in the previous mappings (interfacing patterns, themes, function roles)
Those pages are a bad read. They seem rather separate sentences, often long sentences, and each sentence ends with a reference.
Page 32: “section ??”
Page 32: “same running scenarios used throughout this paper: a manufacturing maintenance copilot (section 8)”?? This first time that copilot is mentioned.
Page 39: I would remove the “Outlook” paragraph. The authors have already a good section Future Directions and Challenges.
What do the authors mean with the field can move toward systems that are dependable systems??
Suggestion: including some quantitative insides from your work would be interesting as well.