Final Project Milestone

Manuscript
Sidhant Subramanian
GTID: 903847536

generated in similar locations or a transformation of a

Abstract location
This is the first submission towards Project Milestone 1 Related Work
We conduct a cognitive task analysis of the ARC Discrete Search
Challenge Tasks to get an understanding of the task and
what an AI system would need to know to solve it. After
Neural-guided, Bidirectional Program Search
that we conduct a Literature Review on problems solving
for Abstraction and Reasoning
ARC challenges and summarize the Findings and
shortcomings Findings
1. The researchers created an innovative
Datasets — https://github.com/fchollet/ARC-A bidirectional, neural-guided algorithm for
Extended version — https://arcprize.org/ Introduction program search that builds upon execution-guided
synthesis by incorporating inverse semantics.
Shortcomings
Introduction 1. The Dream Coder approach used a simple search
By looking at the link provided, solving more than 30 ARC method, which may not work well for more
problems given and brainstorming on why a particular difficult tasks.
output is being generated we can gain an understanding of 2. In the ARC symmetry tasks, using a bidirectional
the Hypothesis, Strategy to apply and some ideas on what approach didn't show any clear advantage over the
an AI system needs to be built. forward-only baseline

• The knowledge used was the input images provided for Domain-Specific Language (DSL) Program
each ARC problem and their corresponding output for each Synthesis
input image. From the input images, we observe the color
and position of each square in the matrix and then observe
the corresponding output that is given.
Program Synthesis using Inductive Logic
Programming for the Abstraction and
Reasoning Corpus
• We focus on a pattern that is followed by observing the Findings
transformation of each square or sometimes a 1. The object-centric approach simplified the search
transformation on a group of squares, mapping them to the process by focusing on specific objects rather than
outputs, and finding reasons as to why a particular output analyzing entire grids.
has been generated. 2. Using multiple overlapping object representations
• Each pattern is unique where some follow simple steps for the same image improved the ability to find
like filling up the squares between other squares, some the right way to make changes.
involve finding how the transformation is occurring based Shortcomings
on the location of squares in the input the output squares 1. The system uses a deductive search to apply
are programs in a specific order that covers the most
areas, but this may not always be the best
approach.
Graphs, Constraints, and Search for the Theoretical Framework & Approach
Abstraction and Reasoning Corpus
Findings Breadth-first search is a strategy of systematically
1. Using constraint acquisition was helpful in narrowing visiting all nodes in a graph or tree by traversing all the
nodes at the current depth prior to moving to nodes at
down the options, which meant we had to look the next depth level.
at 38% fewer possibilities before finding a solution.
It keeps a record of the nodes in a queue, starting with
2. ARGA's way of using graph abstractions seems to the root, and enqueues successors, without allowing
be good at solving complicated tasks that focus on any state to revisit. The successor function generates
objects. possible next states, and then BFS places them into a
3. It is especially successful with the challenging tasks queue in a first-in-first-out manner, exploring all nodes
in the Abstraction and Reasoning Corpus (ARC) at the present depth before moving deeper.
It checks for any node expansion that results in the goal
Shortcomings state.
1. The current implementation is in Python, but the In graph search, in order to avoid cycles, BFS will
results shown are in C++ which can contribute to a never go back to any state that has been visited before;
reduction in performance there is no such restriction in tree search.

Representational Structures
Generalized Planning for the Abstraction and We model all the possible states that are obtained while
Reasoning Corpus applying a particular transformation as a node in the
graph. BFS iterates through all these steps generated
Findings and keeps track of them.
1. GPAR is good at changing the colors of things. It's The various actions are the transition between these
especially good at doing this in a simple way by nodes.
Each of these states are generated by the successor
using descriptions of things like size, shape, and function in BFS.
color.
2. GPAR performs well in different situations, with The algorithm stores the state where the final goal is
very little difference between its train and test reached.
compared to other models. Performance and learning mechanisms
Shortcomings
1. GPAR might need to try out lots of different ideas Once the goal state is found the algorithm stops its
and settings before figuring out the right answer for execution. This rule-based pruning helps in reducing
the depth of the search improving its time complexity
certain jobs. This is shown by the large number of and efficiency.
attempts it makes at tasks that it eventually solves
BFS also keeps track of states it has already visited and
does not continue that track if it has already searched
Ensemble Solutions for to reduce redundancy.
1. Bagging
Using multiple models on different
subsets of data and finally aggregating
Initial Findings
output based on techniques like After running the code locally, I was able to generate a score
majority voting and averaging. of 1.

2. Boosting BFS seemed like the most logical solution at the time since
Using multiple models where a new it generates a tree with all the transformations sequence
model is trained on the errors of the possible between the initial state and the final state and
previous model. running a brute force through all of them guarantees to find
the result.
3. Stacking
Using multiple models where a Since a score of only 1 was achieved I concluded that BFS
model’s outputs are used as inputs to a was working only on transformations that were short and
new model. simple and failing to grasp transformations that were more
complicated in nature due to time or memory limitations.
The algorithm was probably traversing many unnecessary
states along the path of the result since it had no additional
knowledge or metric to make decisions and seemed to be too
simple to generate more accurate results. g(n) = This function quantifies the precise cost associated
with traversing from the starting node to a given node in the
BFS code graph.
def bfs(graph, start_node):
# Initialize a queue for BFS h(n) = A* employs a heuristic function to estimate the cost
queue = deque([start_node]) of the remaining path from the current node to the goal node,
allowing for efficient prioritization of node exploration
# Initialize a set to keep track of based on projected total costs.
visited nodes
visited = set([start_node]) f(n) = The cumulative estimated cost to traverse the path via
node n is considered, with priority given to nodes exhibiting
# While the queue is not empty lower f(n) values.
while queue:
# Dequeue the front node It aims to find the shortest path in terms of cost in
current_node = queue.popleft() environments where the cost to travel between nodes varies.
print(current_node) # Process the
current node (e.g., print it) A* is optimal for finding paths in maps or weighted graphs
where the shortest path is not necessarily the one with the
# Visit each neighbor of the current fewest nodes.
node
for neighbor in Heuristics used by me
graph.get(current_node, []):
if neighbor not in visited: • Chebyshev Distance:
# Add the neighbor to the
queue and mark it as visited
h(n)=max(∣xgoal−xn∣,∣ygoal−yn∣)
queue.append(neighbor)
visited.add(neighbor)
In scenarios where the cost of diagonal movement is
equivalent to that of orthogonal movement, this approach
For future endeavors it is important to improve the
is applicable.
complexity of the algorithm. One of the ways to do this is
by incorporating heuristics in the algorithm.
• Euclidean Distance:
The search function of the BFS can incorporate a heuristic
where the cost to reach a node and the goal is considered. h(n)=(xgoal−xn)^2+(ygoal−yn)^2
Based on the value calculated with this heuristic prioritizing
these states over others can improve the search time. This framework is applicable in continuous 2D domains or
grid-like structures where diagonal movements are
Previously solved problems can be stored and reused later permitted and assigned varying costs.
during the search to provide further optimization of the
algorithm.
Results
The heuristic function can also be optimized based on
analyzing the results generated. Algorithm Heuristics Score

BFS None 1
Methodology
A* Chebyshev 1.5
A* is a best-first search algorithm that uses both the actual Distance
cost from the start node to the current node and an estimated
cost from the current node to the goal (heuristic). A* Euclidean 2.5
Distance
f(n)=g(n)+h(n)
 Conclusion
References
The findings from this milestone illustrate the limitations
and potential of different search strategies in solving Wikipedia contributors. (2024c, November 26). A* search
transformation problems. Breadth-First Search (BFS) was algorithm. Wikipedia.
initially employed due to its systematic approach to https://en.wikipedia.org/wiki/A*_search_algorithm
traversing all possible states. However, BFS yielded a low
score of 1, demonstrating inefficiency in handling complex ARC Prize 2024. (n.d.-b). Kaggle.
transformations due to its lack of heuristics and inability to https://www.kaggle.com/competitions/arc-prize-2024/code
prioritize paths, resulting in unnecessary state explorations.
Incorporating the A* search algorithm with heuristic ARC Prize - Official Guide. (n.d.-b). ARC Prize.
functions, such as Chebyshev and Euclidean distances,
showed improvement by achieving scores of 1.5 and 2, https://arcprize.org/guide
respectively. This suggests that heuristic-based approaches
offer a more efficient solution by guiding the search towards Program Synthesis using Inductive Logic Programming for
the goal state and reducing redundancy. the Abstraction and Reasoning Corpus. (n.d.-b). Ar5iv.
https://ar5iv.labs.arxiv.org/html/2405.06399v1
The results from this milestone highlight both the
constraints and advantages associated with various search Zenkner, J., Dierkes, L., Sesterhenn, T., & Bartelt, C. (2024,
strategies for addressing transformation problems. Initially, May 27). AbstractBeam: Enhancing Bottom-Up Program
Breadth-First Search (BFS) was applied due to its Synthesis using Library Learning.
exhaustive nature in exploring all possible states. However, https://export.arxiv.org/abs/2405.17514
BFS demonstrated significant inefficiency, achieving a low
performance score of 1. This can be attributed to its non- Machine Learning Street Talk. (2024b, June 18). Chollet’s
heuristic framework and its failure to prioritize exploration ARC Challenge + Current Winners [Video]. YouTube.
paths, leading to excessive and unnecessary state https://www.youtube.com/watch?v=jSAT_RuJ_Cg
evaluations.

In contrast, the integration of the A* search algorithm with

heuristic functions—specifically Chebyshev and Euclidean
distances—yielded notable improvements, producing scores
of 1.5 and 2, respectively. These findings indicate that
heuristic-driven approaches can enhance search efficiency
by directing the exploration towards the goal state while
minimizing redundant computations.

Despite the advancements made, the outcomes continue to

fall short of expectations. Next steps involve honing
heuristic functions, visualizing state transitions to pinpoint
where deviations arise, and adaptively adjusting heuristics
based on observed trajectories. Furthermore, investigating
deep learning methodologies, particularly Large Language
Models (LLMs), could significantly boost performance by
utilizing cutting-edge approaches to tackle the task's
inherent complexities.