Skip to content

2023-04-06T06:32

https://ai.facebook.com/research/publications/segment-anything/

2023-06-01T16:13

  • Mis en page. Le texte de la preuve n'est pas aligne avec les images.
  • Write more about the chan vese model. Show how the curvature term can change the results.
  • The data model is very simple. There is no curvature term. The paper was a classic Chan-Vese model.

2023-06-15T18:10

About my quick study on the Chan-Vese alike data term for Graph Flow

I re-read the code I wrote for the Chan-Vese model of Graph Flow to evaluate if we should keep, extend or remove the section we dedicated to it in the paper.

Summary of the model

  1. The initial segmentation is an array of disks. They cover all the input image and are equally spaced vertically and horizontally. They compose what is called in the code a disk mask. Parameters are available.
  2. In each iteration, the following is executed:
  3. For each connected component, we compute its RandomNeighborhood. It is composed of the shape itself, one dilation, one erosion and then n random morphological operations in the contour of the connected component.
  4. For each connected component, we build the graph and find its minimum cut for EACH ONE if its neighbors. These form the candidates for this connected component.
  5. After we have computed the candidates for all connected components, we are going to select the best candidate of each connected component to generate a candidate solution.
  6. We compute the energy for the candidate solution and if it is smaller than the current one, we select the candidate, update the solution and re-start the process. Otherwise, we repeat the process above.
  7. If after 20 times this process is executed no best candidate solution was found, we stop the process.

Remarks

  1. We have parameters to control the candidate generation and the candidate selection. In particular, the coefficients used in the graph controls candidate generation and the coefficients used to compute the energy controls candidate selection. Therefore, I can have a parameter configuration that generates candidates that are more aligned with the curvature term but then, during candidate selection, the data term is preferred over elastica.
  2. I thought of creating a unique graph that would take into account all connected components at once. My idea was that this could favor the computation of the Chan-Vese data term. The curvature weights would still be computed isolated, component by component, to avoid the effect of having the estimator disk touching different components at the same time. To take full advantage of this, I would have to add more edges by nodes.
  3. I thought about SuperPixels in order to handle scenarios in which the estimator disk is too big to estimate curvature. This is something that could be handle. This would enable us to correctly estimate curvature even with low resolution images. For example, to apply our model to segment thin lines, of one pixel. We could use SuperPixels to represent the line, and then we could subdivide them (every time a pixel is divided by 4). I believe this could be done in an efficient way, such that only the original pixels in the line are transformed in SuperPixels. Of course, there would be a limit for it. We cannot subdivide a SuperPixel indefinitely. But even for images with 1 pixel long lines, a subdivision by 1/16 would be more than sufficient to do precise estimations.
  4. In the GraphFlow, we should maintain the state of a SuperPixel and come up with a criteria to remove or add the SuperPixel as a normal pixel. I believe that the "All-or-nothing" criteria is the most appropriated. If all SubPixels are activated, the SuperPixel is a valid Normal Pixel; if all SubPixels are inactive, the normal pixel is removed.
  5. Normalization factor. It is super important to have a criteria to normalize the Data and Energy terms. The criteria I come up with is the following. nf = Energy(standard_configuration) / Data(standard_configuration). As standard configuration I mean something as "all-parameters-equal-to-one" or something like that.

2023-06-15T18:23

Hot to improve the Chan-Vese alike model

  1. Implement the SuperPixel evaluation.
  2. Create a single graph for all the connected components. The curvature term is evaluated individually though, still using the optimization band. The data term, on the other hand, is computed for all the pixels.
  3. Define standard experiments and create scripts that generate reports. These tools are important to make a sensibility analysis on the parameters and gain intuition on how the model works.

2023-06-16T16:32

Perimeter influence

During the discussion with Jaco, we asked ourselves about the influence of the perimeter in our model. Jaco suggested to regenerate the experiment with the images of the Coco dataset. Unfortunately, would not be that easy to do that. I lost the gc-object.xml of the input images. They are necessary to automatically execute the script. What I do have are the seeds used, but they were not saved as binary masks. They are drawn in the image itself using a single color. I would have to create something to extract the masks from the seed images, convert them into binary masks and then convert them into gc-object.xml. My understanding is that the perimeter has very little influence in our model. The perimeter is only used in the candidate selection part. That is, when we compute the elastica to select the best candidate. One way to analyse the influence of the perimeter is to use different values for b and B, that is, the value I use for the curvature in the computation of the Elastica and in the computation of the balance coefficient. The idea would be to set B=1 and b=0. Therefore, I would still be producing candidates based on the balance coefficient flow, but the selection would be based on the perimeter itself and not the curvature.

2023-06-16T17:14

Model description:

  1. CSF + Data term to generate candidates
  2. Elastica to select the best candidate

The model assumes that CSF applied to a shape neighborhood can produce shapes with lower Elastica value than the original image. Since we can validate the Elastica value with our digital estimator, we are in a position to select the best candidate.