Largest row-wise and column-wise sorted sub-matrix

Given an N * M matrix mat[][], the task is to find the area-wise largest rectangular sub-matrix such that each column and each row of the sub-matrix is strictly increasing. 

Examples:  

Input: mat[][] = 
{{1, 2, 3}, 
{4, 5, 6}, 
{1, 2, 3}} 
Output: 6 
Largest sub-matrix will be {{1, 2, 3}, {4, 5, 6}}. 
Number of elements in this sub-matrix = 6.

Input: mat[][] = 
{{1, 2, 3}, 
{4, 5, 3}, 
{1, 2, 3}} 
Output: 4 
The largest sub-matrix will be 
{{1, 2}, {4, 5}} 

Approach: There are many approaches to solve this problem varying from O(N³ * M³) to O(N * M). In this article, an approach with O(N * M) time complexity using a stack will be discussed. 
Before proceeding further, its recommended to solve this. problem.

Let's try to understand the approach broadly, then the algorithm will be discussed. For every column of the matrix, try to find the largest row-wise and column-wise sorted sub-matrix having the left edge at this column. To perform the same, create a matrix pre[][] where pre[i][j] will store the length of the longest increasing sub-array starting from the index j of the array arr[i].
Now using this matrix, for each column j, find the length of the longest row-wise and column-wise sorted array. To process a column, all the increasing sub-segments of the array pre[][j] will be required. The same can be found using the two-pointer technique. In each of these sub-segments, simply find the largest area under the histogram considering the row-wise increasing sub-segments as bars. 

• Create a prefix-sum array for each row 'i', which stores length of the largest increasing sub-array ending at each column 'j' of that row.
• Once we have this array, for each column 'j'. 
  
  • Initialize 'i' equals 0.
  • Run a loop on 'i' while 'i' is less than 'N' 
    
    • Initialize 'k' equals i+1.
    • while k less than N and arr[k][j] greater than arr[k-1][j], increment k.
    • Apply histogram problem on the sub-array pre[i][j] to pre[k-1][j], to find the largest area under it. Let us call this value 'V'. Update final answer as ans = max(ans, val).
    • Update 'i' equals k-1.

Below is the implementation of the above approach:  

6

Time Complexity: O(N*N), as we are using nested loops to traverse N*N times.

Auxiliary Space: O(N*N), as we are using extra space for matrix.