Dynamic Programming Approach: A Brief Analysis

Dynamic Programming Approach: A Brief Analysis

It is possible that you were discouraged by the term “dynamic programming”. Usually, you can tell a lot about the algorithms from their makes. For example, recursion, backtracking, divide and conquer, and greedy algorithms, you can get a general idea from the names. However, dynamic programming is something else. Hence, we can not identify it by its name. In fact, it is possible to misunderstand it by its confusing name. To clarify this concept, algo.monster shares this article to help you. Hopefully, you will get a better understanding of dynamic programming thinking.

Dynamic programming (DP) is an important element in the introduction to algorithms. It’s an optimization algorithm that has important applications in many fields including machine-learning algorithms.

Dynamic programming and operations research

Let’s first discuss the topic of “operations research”. It is an applied discipline in mathematics that employs statistics and mathematical models. It can solve complex problems by finding optimal or close-optimal solutions. In reality, operations Research is able to solve real-life problems. Also, it can significantly increase or optimize the efficiency of the existing systems.

Well, dynamic programming actually exists as a branch of operations research. And it serves as an optimization method in many applications. This DP method converts a multi-stage decision-making process into several single-stage problems. And you can solve these simpler stages directly one by one.

This idea is a result of American mathematician Richard Bellman’s research on multi-stage decision problem-solving. Then, he created the concept of dynamic programming. Richard Bellman said that we shouldn’t speculate on details other than its name. When applying for grants from the government, he and his staff chose to conceal certain facts.

Get to know the basic idea of dynamic programming

First of all, note that dynamic programming is not a name for its face value. That is because its name does not indicate its core idea. In fact, it is better to name your program after the idea. The names we’re talking about are those like the recursive formula, state transfer equation, or other names.

Second, dynamic programming is more than just an algorithm. Instead of taking it as a single algorithm, it’s better to see it as a problem-solving methodology.

The third is to find the forms of the optimal value. Generally, the longest subsequence or the maximum segment and the optimal binary search tree are three of the common forms.

To solve the original problem, you need to break it down into subproblems. Then, the subproblems that you get by decomposing the problem for dynamic programming are not always independent. Well, this is a different approach to the partitioning method. As an example, let’s say that the partitioning method is able to solve such problems. The problem is then so complex that you get so many subproblems that it takes an exponentially long time to deal with.

The core idea of DP 

Some subproblems can be solved multiple times using the partitioning method. Let’s say we could save the solutions to subproblems that have been solved and then find them again when necessary. This will save us a lot of repetitions and allow us to obtain an algorithm with polynomial complexity. The table can store all sub-problems that were solved to reach the sub-purpose. The computed table will contain the results, regardless of whether the subproblem was used later. This is the core idea.

How DP solves problems?

To avoid repeat computations, the dynamic programming algorithm first divides the problem into subproblems. Then, it saves the answers to the subproblems. Finally, the original problem is solved by the combination of solutions to the subproblems.

To determine if a problem can or cannot be solved using dynamic programming, there are two elements that are essential: the “optimal structure-property” and “overlapping subproblem properties”.

dynamic programming overlapping subproblems

Optimal substructure property

The best way to design a dynamic programming algorithm is to identify the optimal solution structure. The optimal substructure property of a problem is when it contains the optimal solutions to its subproblems. The optimal substructure property is a key indicator of how DP can solve the problem.

Overlapping subproblem property

A recursive algorithm is when you solve a problem top-down. Also, you may compute some subproblems multiple times. This algorithm takes advantage of the overlapping subproblem property and solves each subproblem once, saving the solution to a table. We can see the final result in constant time when we solve the subproblem again.

Dynamic programming and partitioning method

Sometimes, subproblems arising from the decomposition dynamic programming are dependent on one another. As an example, let’s say that the subproblems decomposed are independent, like Binary Search. The decomposed subproblems will be independent in this case. So, it does not have an overlapping subproblem. As a result, this makes it unsuitable for dynamic programming. However, it is more suitable for partitioning algorithms in this case. Actually, the Fibonacci series problem is more applicable to DP. But the Fibonacci series solution is not strictly a dynamic programming thinking method. It is, however, a good example of how to use it. After all, the Fibonacci numbers help you have a better understanding of “overlapping subproblem properties”.

Conclusion 

Normally, people regard dynamic programming algorithms as a way to solve problems efficiently. Yet, it is more than just an algorithm, it provides ideas of mythology. To get a better understanding of DP, you need to learn about the optimal substructure and the overlapping subproblems.

One funny thing as we’ve mentioned, the name dynamic programming itself doesn’t show the exact meaning of the algorithm. Although the creator chose the name originally, it’s actually misleading on many levels. To better define it, I recommend the words from this book.

Quote: “The term dynamic programming refers to a collection of algorithms which can be used to compute optimal policies given a perfect model of the environment as a Markov decision process.” (Quotes from the book Reinforcement Learning: An Introduction)

Or else, you can check more information from Wikipedia or algo.monster.

We will be happy to hear your thoughts

Leave a reply

SEO Article Zone
Logo
Reset Password