Approximation algorithms for Knapsack

Knapsack problems are those problems in which some set of items will be given to us, each with a weight and value and we will be asked to find the most valuable combination by maximizing the total value of items and the weight should not exceed the knapsack weigh.

👁 file

Approximation Algorithms:

It plays a vital role in finding the optimal solution to the knapsack problems as in real-world scenarios finding the exact optimal solution to the knapsack problem is quite impractical due to the problem's NP-hard nature. These approximation algorithms offer a reasonable solution to the knapsack problem by taking both time and space complexity into consideration.

we will discuss two approximation algorithms:

• Greedy Algorithms
• Dynamic Programming

Let's consider a problem statement to understand these algorithms better:

Given a set of items, each with a weight and a value, and a knapsack of limited capacity, we need to find the most valuable combination of items to include in the knapsack while ensuring that the total weight does not exceed the knapsack's capacity.

Examples:

Input: N = 3, W = 4, values[] = {1, 2, 3}, weight[] = {4, 5, 1}
Output: 3
Explanation: In this example, we have three items with respective values and weights:

• Item 1: value = 1, weight = 4
• Item 2: value = 2, weight = 5
• Item 3: value = 3, weight = 1

The knapsack has a capacity of 4. By selecting either Item 1 or Item 3, we can achieve the maximum value of 3. Since the weight of Item 1 (4) exceeds the knapsack's capacity, we choose Item 3, which has a weight of 1 and a value of 3.

Input: N = 3, W = 3, values[] = {1, 2, 3}, weight[] = {4, 5, 6}
Output: 0

Greedy Algorithms to the Knapsack Problem

The greedy approach is a simple and intutive algorithm for solving knapsack problem. It will select the items based on theri value to weight ratios and choosing the items with highest ratios first.

Step-by-step algorithm:

• Sort the items in descending order of their value-to-weight ratios.
• Initialize the knapsack as empty and set the total value to zero.
• Iterate through the items in the sorted order:
  • If the current item can be fully included in the knapsack, add it completely and update the total value.
  • Otherwise, include a fractional part of the item that fits the remaining capacity of the knapsack, proportionally increasing the total value.
• Return the knapsack's final configuration and the total value.

Below is the implementation of the above approach in Python:

3

Time Complexity: O(N log N) where N is the number of items due to sorting
Auxiliary Space: O(N) where N is the number of items. 

Dynamic Programming Approach for the Knapsack Problem

Using dynamic programming we can break down the problem into smaller subproblems and will use a table to store the optimal solutions for the these subproblems. We will iterate through each item and weight combination making a decision to either include or exclude the item based on its value and weight and we can achieve result by avoiding redundant calucations

Step-by-step algorithm:

• Create a 2D list called dp with dimensions (N+1) x (W+1) and initialize all values to 0. This table will store the maximum achievable values for different combinations of items and capacities.
• Iterate through each item from 1 to N and each capacity from 1 to W:
  • If the weight of the current item is greater than the current capacity, it cannot be included in the knapsack. So, assign the value at the previous item and the same capacity to dp[i][j].
  • If the weight of the current item is less than or equal to the current capacity, we have two choices:
  • Include the current item: Add its value to the value obtained by considering the remaining capacity after including the item (values[i-1] + dp[i-1][j - weights[i-1]]).
  • Exclude the current item: Consider the value obtained by excluding the item (dp[i-1][j]). Choose the maximum value between the two choices and assign it to dp[i][j].
• After completing the iterations, the value at dp[N][W] represents the maximum achievable value for the given knapsack capacity.
• Return the value dp[N][W] as the maximum value that can be obtained.

Below is the implementation for the above approach:

3

Time Complexity: O(N * W) where N is items and W is capacities.
Auxiliary Space: O(N * W) where N is items and W is capacities.