A small gap between the pre-training model you provide and your paper in the loop of the RobotCar dataset

[zhuixunforever]

Thank you for your work!
I download your pre-training model and eval it by evat.py

This is LOOP2(2014-06-23-15-36-04) output:

Error in translation: median 4.7186 m, mean 8.5358 m
Error in rotation: median 3.3001 degrees, mean 4.5155 degree

LOOP1(2014-06-23-15-41-25)

Error in translation: median 5.0281 m, mean 8.5667 m
Error in rotation: median 3.2837 degrees, mean 4.6280 degree

but in your paper, it's:
median_t | median_r | mean_t | mean_r |
LOOP2 | 5.05 | 2.01 | 8.86 | 4.67 |
LOOP1 | 5.68 | 2.23 | 8.61 | 4.58 |

There seems to be a little bit of a smile gap in median result.

But when I eval 'stairs' , it is the same:

Error in translation: median 0.2638 m, mean 0.3076 m
Error in rotation: median 10.5399 degrees, mean 10.5156 degree

in your paper:
median translation and rotation is: 0.26 | 10.5

Can you give me some advice, thank you a lot.

[BingCS]

Hi,

Thanks for your question. Do you test the code with pytorch 0.4.1+cuda 9.1 on RTX 2080ti? Previously, I tested my code on our new server (RTX 2080ti) instead of the old one (Titan V and GTX 1080), and 'THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument' will be displayed. Although the code still report results, they are slightly different from the original. Similar to you, the difference on RobotCar can be directly observed, while that on 7Scenes can only be seen with more significant digits. As the difference of results comes out with such error message, it might be affected by the environment with 2080ti. Please refer to pytorch/pytorch#15797 for more information.

Best,
Bing

[zhuixunforever]

Yeah, I use pytorch 0.4.1 on RTX 2080ti, with cuda 10 or cuda9.0, get a similar result. The same, I get 'THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument'. Therefore, this should be the problem of 2080ti.But I only have 2080Ti, I can't change other GPU. Have you solved this problem in 2080ti?

Thank you for your reply.

[BingCS]

Sorry for disappointing you, I don't spare time on solving this problem recently. I just keep using the old configuration.

[zhuixunforever]

I saw someone solve it by “pip3 install -U https://download.pytorch.org/whl/cu100/torch-1.0.0-cp36-cp36m-linux_x86_64.whl”, and I will try it by "pip3 install -U https://download.pytorch.org/whl/cu90/torch-0.4.1-cp27-cp27m-linux_x86_64.whl". After I try it, I'll tell you the result.

[zhuixunforever]

Just installing pytorch0.4.1 by pip from https://download.pytorch.org/whl/cu90/torch-0.4.1-cp27-cp27m-linux_x86_64.whl didn't work for me.

I find someone solve the compatibility problem between 2080Ti and Pytorch0.4.1 by installe cuda9.2, I try it. And 'THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument' didn't display, like this:

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
------------ Options -------------
batchsize: 64
beta: -3.0
color_jitter: 0.7
cropsize: 256
data_dir: ./data
dataset: RobotCar
epochs: 100
exp_name: name
gamma: None
gpus: [0]
logdir: ./logs
lr: 5e-05
lstm: False
model: AtLoc
models_dir: models
nThreads: 8
print_freq: 20
real: False
results_dir: figures
runs_dir: runs
save_freq: 5
scene: loop
seed: 7
sei: 3.0
skip: 10
steps: 3
test_dropout: 0.0
train_dropout: 0.5
val: False
val_freq: 5
variable_skip: False
weight_decay: 0.0005
weights: ./logs/RobotCar_loop_AtLoc_False/models/epoch_300.pth.tar
-------------- End ---------------
/home/ar443/A_project/B_software_VisualLocalization/AtLoc/data/robotcar_sdk/interpolate_poses.py:143: RuntimeWarning: divide by zero encountered in divide
(pose_timestamps[upper_indices] - pose_timestamps[lower_indices])
Loaded weights from ./logs/RobotCar_loop_AtLoc_False/models/epoch_300.pth.tar
Image 0 / 3356
Image 200 / 3356
Image 400 / 3356
Image 600 / 3356
Image 800 / 3356
Image 1000 / 3356
Image 1200 / 3356
Image 1400 / 3356
Image 1600 / 3356
Image 1800 / 3356
Image 2000 / 3356
Image 2200 / 3356
Image 2400 / 3356
Image 2600 / 3356
Image 2800 / 3356
Image 3000 / 3356
Image 3200 / 3356
Error in translation: median 5.0281 m, mean 8.5277 m
Error in rotation: median 3.2837 degrees, mean 4.6104 degree

But there is still a gap, comparsion with your paper
LOOP1 | 5.68 | 2.23 | 8.61 | 4.58 |

There is only one RuntimeWarning in interpolate_poses.py , as mentioned above,
Do you think that's the effect?

[zhuixunforever]

The reason of RuntimeWarning is, the max timestamps of ins.csv large the max timestamps of stereo.timestamps in LOOP1(2014-06-23-15-41-25). I tried to remove the extra timestamps in stereo.timestamps, but that didn't work.

The final output is:

Loaded weights from ./logs/RobotCar_loop_AtLoc_False/models/epoch_300.pth.tar
Image 0 / 3343
Image 200 / 3343
Image 400 / 3343
Image 600 / 3343
Image 800 / 3343
Image 1000 / 3343
Image 1200 / 3343
Image 1400 / 3343
Image 1600 / 3343
Image 1800 / 3343
Image 2000 / 3343
Image 2200 / 3343
Image 2400 / 3343
Image 2600 / 3343
Image 2800 / 3343
Image 3000 / 3343
Image 3200 / 3343
Error in translation: median 5.0030 m, mean 8.5316 m
Error in rotation: median 3.2781 degrees, mean 4.6140 degree

I can't solve it. That's too bad

[zhuixunforever]

By the way, do you undistorts the images when you train or test? In eval.py, it seems to call 'centre' instead of 'centre_processed' that is undistorted images. And when I use 'centre_processed', it output worse results than 'centre'.
I have this problem because in your README.md, you say

To make training faster, we pre-processed the images using data/process_robotcar.py.

[male-lion]

Thank you for your work.
I have some professional questions to ask you.
This is my mailbox, 282117590@qq.com.
I think there will be a pleasant exchange between us
Please contact me.