Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add tests for chessboard corner detection #52

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
+74 −0
Open

add tests for chessboard corner detection #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5753120
add tests for chessboard corner detection
yakir12 Dec 24, 2024
c82909c
trying to pass CI with a silly repetition of the corner detection
yakir12 Dec 24, 2024
38ab7e3
trying to pass CI by adding an initial corner detection
yakir12 Dec 24, 2024
6f782a2
Update test/test_corner_detection.jl
yakir12 Dec 29, 2024
8c4babb
address tim's points
yakir12 Dec 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions test/Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,7 @@ Artifacts = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
Downloads = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
FileIO = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
LazyArtifacts = "4af54fe1-eca0-43a8-85a7-787d91b784e3"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
OpenCV = "f878e3a2-a245-4720-8660-60795d644f2a"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
XML = "72c71f33-b9b6-44de-8c94-c961784809e2"
3 changes: 3 additions & 0 deletions test/runtests.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ using LazyArtifacts
using OpenCV
using FileIO
using Test
using XML
using LinearAlgebra

if "OPENCV_TEST_DATA_PATH" in keys(ENV)
test_dir = joinpath(ENV["OPENCV_TEST_DATA_PATH"], "cv")
Expand All @@ -18,4 +20,5 @@ end
include("test_objdetect.jl")
include("test_dnn.jl")
include("test_fileio.jl")
include("test_corner_detection.jl")
end
69 changes: 69 additions & 0 deletions test/test_corner_detection.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
const Point = Tuple{Float32, Float32} # just for conviniencve

function detect_corners(file, n_corners)
img = load(file)
gry = img[1:1, :, :]
cv_n_corners = OpenCV.Size{Int32}(n_corners...)
_cv_corners = OpenCV.Mat(Array{Float32}(undef, 2, 1, prod(n_corners)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, could this be declared OpenCV.Mat(Matrix{Tuple{Float32,Float32}}(undef, n_corners))? And then avoid the reshaping/eachslice below?

Copy link
Author

@yakir12 yakir12 Dec 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, OpenCV.Mat can only accept 3D arrays, not matrices:

  MethodError: no method matching OpenCV.Mat(::Matrix{Tuple{Float32, Float32}})
  The type `OpenCV.Mat` exists, but no method is defined for this combination of argument types when trying to construct it.
  
  Closest candidates are:
    OpenCV.Mat(::AbstractArray{T, 3}) where T<:Union{Float32, Float64, Int16, Int32, Int8, UInt16, UInt8}
     @ OpenCV ~/.julia/artifacts/dcc70861ed0ffc5427898f6aeb436620f33fa02f/OpenCV/src/Mat.jl:13

So that's not an option.

Copy link
Member

@timholy timholy Dec 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to do

corners = Matrix{Tuple{Float32,Float32}}(undef, n_corners)
ret, cv_corners = OpenCV.findChessboardCorners(gry, cv_n_corners, OpenCV.Mat(reinterpret(reshape, Float32, corners)), 0)
@assert ret "Failed to detect any corners!"
return corners

if you think that's a little bit clearer/cleaner. It certainly makes inference's job easier.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is clearer, does not error (and TIL about reinterpret(reshape...), and I would love to use it, but the call to findChessboardCorners does not mutate its arguments:

corners = Matrix{Tuple{Float32,Float32}}(undef, n_corners)
before = deepcopy(corners)
ret, cv_corners = OpenCV.findChessboardCorners(gry, cv_n_corners, OpenCV.Mat(reinterpret(reshape, Float32, corners)), 0)

julia> all(corners .=== before)
true

So we would anyways need to reshape and slice the return cv_corners value, like this:

function detect_corners(file, n_corners)
    img = load(file)
    gry = img[1:1, :, :]
    cv_n_corners = OpenCV.Size{Int32}(n_corners...)
    _corners = Matrix{Tuple{Float32,Float32}}(undef, n_corners)
    ret, cv_corners = OpenCV.findChessboardCorners(gry, cv_n_corners, OpenCV.Mat(reinterpret(reshape, Float32, _corners)), 0)
    corners = reshape(Point.(eachslice(cv_corners, dims = 3)), n_corners)
    return corners
end

I've noticed this issue before (cause I tried something similar to what you suggested). This is surly another bug..?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must be. Otherwise why pass an array pre-allocated for the output?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand why you'd think so, I did too, but lo and behold:

julia> hash(_corners)
0x88d97a55d4ebfe14

julia> before = deepcopy(_corners)
7×5 Matrix{Tuple{Float32, Float32}}:
 (1.0f-45, 0.0)  (0.0, 0.0)      (1.0f-45, 0.0)  (0.0, 0.0)      (0.0, 0.0)
 (3.0f-45, 0.0)  (6.0f-45, 0.0)  (0.0, 0.0)      (1.0f-45, 0.0)  (0.0, 0.0)
 (0.0, 0.0)      (1.0f-45, 0.0)  (1.0f-45, 0.0)  (0.0, 0.0)      (0.0, 0.0)
 (1.0f-45, 0.0)  (0.0, 0.0)      (1.0f-45, 0.0)  (1.0f-45, 0.0)  (1.0f-45, 0.0)
 (1.0f-45, 0.0)  (1.0f-45, 0.0)  (1.0f-45, 0.0)  (0.0, 0.0)      (1.0f-45, 0.0)
 (4.0f-45, 0.0)  (0.0, 0.0)      (1.0f-45, 0.0)  (3.0f-45, 0.0)  (0.0, 0.0)
 (1.0f-45, 0.0)  (0.0, 0.0)      (0.0, 0.0)      (1.0f-45, 0.0)  (0.0, 0.0)

julia> ret, cv_corners = OpenCV.findChessboardCorners(gry, cv_n_corners, OpenCV.Mat(reinterpret(reshape, Float32, _corners)), 0)
(true, Float32[89.02004; 65.43221;;; 101.30311; 66.39866;;; 114.23407; 67.06309;;; … ;;; 153.37643; 118.27199;;; 166.67719; 119.35007;;; 180.67068; 120.32631])

julia> hash(_corners)
0x88d97a55d4ebfe14

julia> all(_corners .=== before)
true

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what is that argument for?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know and I don't know how to find out, everything is wrapped here. I suppose we could turn a blind eye and just follow their example:

ret, corners = cv.findChessboardCorners(img, cv.Size{Int32}(7,5))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least the 2-arg form doesn't pretend to be passing in the output array, so I'd go with that if you can't figure out the source of the problem. But from the OpenCV C++ docs for findChessboardCorners it clearly is supposed to be the output array.

ret, cv_corners = OpenCV.findChessboardCorners(gry, cv_n_corners, _cv_corners, 0)
@assert ret "Failed to detect any corners!"
corners = reshape(Point.(eachslice(cv_corners, dims = 3)), n_corners)
return corners
end

function parse_corners_file(file)
open(file, "r") do o
readuntil(o, "rows:")
rows = parse(Int, readuntil(o, "\n"))
readuntil(o, "cols:")
cols = parse(Int, readuntil(o, "\n"))
readuntil(o, "[")
txt = readuntil(o, "]")
num = parse.(Float32, split(txt, ','))
corners = reverse(permutedims(Point.(eachslice(reshape(num, 2, cols, rows), dims=(2,3)))); dims = 1)
return corners, (rows, cols)
end
end

function get_list(file)
doc = read(file, LazyNode)
str = filter(≠('"'), simple_value(doc[end][end]))
list = Dict{String, String}()
for line in split(str, '\n')
img_file, data_file = split(line)
list[img_file] = data_file
end
return list
end

function calc_error(ps1, ps2)
s = 0.0
for (p1, p2) in zip(ps1, ps2)
s += LinearAlgebra.norm_sqr(p1 .- p2)
end
sqrt(s/length(ps1))
end

function calc_error(img_file::AbstractString, data_file::AbstractString)
corners, n_corners = parse_corners_file(data_file)
detected_corners = detect_corners(img_file, n_corners)
calc_error(corners, detected_corners)
end

@testset "detecting corners" begin

path = joinpath(test_dir, "cameracalibration")
list = get_list(joinpath(path, "chessboard_list.dat"))

k, v = first(list)
calc_error(joinpath(path, k), joinpath(path, v)) # why do we need this?

@testset "in $k" for (k, v) in list

img_file = joinpath(path, k)
data_file = joinpath(path, v)

@test calc_error(img_file, data_file) < 1

end
end
Loading