8000
大佬您好, 我最近在复现您的这篇文章,我使用您提供的配置文件在8张T4上训练dancetrack,每个epoch时长竟然需要2天左右,这是正常的吗? 配置文件如下: --meta_arch motr_unincost --dataset_file e2e_dance --epoch 20 --with_box_refine --lr_drop 8 --lr 2e-4 --lr_backbone 2e-5 --pretrained /pretrained/r50_deformable_detr_plus_iterative_bbox_refinement-checkpoint.pth --batch_size 1 --sample_mode random_interval --sample_interval 10 --sampler_lengths 5 --merger_dropout 0 --dropout 0 --random_drop 0.1 --fp_ratio 0.3 --query_interaction_layer GQIM --num_queries 60 --append_crowd --use_checkpoint --mot_path /code/CO-MOT/data --match_type gmatch --g_size 3
部分训练log如下:
我用v100是训一个epoch大概11个小时,第一个epoch会慢一些
感谢! 请问使用detrex版本的CO-MOT速度会快点吗? 我看那个batch_size可以自己设定,是否可以公布训练代码?我使用那个版本的代码重新训练后测试无结果.
那个我只是简单写了batch_size>1的代码,简单测试了一下没啥大问题,就递交了。不过你可以参考看看有啥我没发现的问题。
好嘞! 感谢!我再研究下!
您好,我全部对照了这个版本的CO-MOT和detrex的CO-MOT代码,感觉是什么问题的,训练的过程好像也对,loss都是正常的,但是训练出来的模型评估结果一直为空,请问可能是什么原因呢?我的配置文件如下:
训练的命令为:
评估结果如下:
为空啥意思,就是没有识别到?
你可以先跑一下我release的权重,看看代码有啥问题
对的, 我按照这个配置文件重新训练然后评估,输出文件每帧都是空的,如下图: , 我用您release的权重直接测试,HOTA在dancetrack val上比论文低4.2%,但是可以跑通. 之后我又对比了各个主要文件,好像都没啥问题. 但训练就是不行😂
那个我只是简单写了batch_size>1的代码,简单测试了一下没啥大问题,就递交了。不过你可以参考看看有啥我没发现的问题。 您好,我全部对照了这个版本的CO-MOT和detrex的CO-MOT代码,感觉是什么问题的,训练的过程好像也对,loss都是正常的,但是训练出来的模型评估结果一直为空,请问可能是什么原因呢?我的配置文件如下: 训练的命令为: 评估结果如下:
在使用detrex训练的时候,直接用那个代码会报错,错误及我的修改如下:
TypeError: repr() got an unexpected keyword argument '_repr_indent' 我将之修改为:
MS- deformable-attension报错 我将之改为:
形参问题: 我将之改为:
函数缺失 我将本仓库的merge函数加入进去:
至此,可以成功训练. 我使用的预训练参数是detrex中Deformable-DETR-R50 + Box-Refinement版本, 在加载的时候有很多不匹配的层: WARNING [04/09 18:46:39 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: class_embed.0.{bias, weight} class_embed.1.{bias, weight} class_embed.2.{bias, weight} class_embed.3.{bias, weight} class_embed.4.{bias, weight} class_embed.5.{bias, weight} position.weight position_offset.weight query_embed.weight query_embed_offset.weight track_embed.linear1.{bias, weight} track_embed.linear2.{bias, weight} track_embed.linear_feat1.{bias, weight} track_embed.linear_feat2.{bias, weight} track_embed.norm1.{bias, weight} track_embed.norm2.{bias, weight} track_embed.norm_feat.{bias, weight} track_embed.self_attn.out_proj.{bias, weight} track_embed.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.class_embed.0.{bias, weight} transformer.decoder.class_embed.1.{bias, weight} transformer.decoder.class_embed.2.{bias, weight} transformer.decoder.class_embed.3.{bias, weight} transformer.decoder.class_embed.4.{bias, weight} transformer.decoder.class_embed.5.{bias, weight} transformer.decoder.layers.0.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.0.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.0.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.0.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.0.linear1.{bias, weight} transformer.decoder.layers.0.linear2.{bias, weight} transformer.decoder.layers.0.norm1.{bias, weight} transformer.decoder.layers.0.norm2.{bias, weight} transformer.decoder.layers.0.norm3.{bias, weight} transformer.decoder.layers.0.self_attn.out_proj.{bias, weight} transformer.decoder.layers.0.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.1.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.1.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.1.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.1.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.1.linear1.{bias, weight} transformer.decoder.layers.1.linear2.{bias, weight} transformer.decoder.layers.1.norm1.{bias, weight} transformer.decoder.layers.1.norm2.{bias, weight} transformer.decoder.layers.1.norm3.{bias, weight} transformer.decoder.layers.1.self_attn.out_proj.{bias, weight} transformer.decoder.layers.1.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.2.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.2.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.2.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.2.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.2.linear1.{bias, weight} transformer.decoder.layers.2.linear2.{bias, weight} transformer.decoder.layers.2.norm1.{bias, weight} transformer.decoder.layers.2.norm2.{bias, weight} transformer.decoder.layers.2.norm3.{bias, weight} transformer.decoder.layers.2.self_attn.out_proj.{bias, weight} transformer.decoder.layers.2.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.3.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.3.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.3.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.3.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.3.linear1.{bias, weight} transformer.decoder.layers.3.linear2.{bias, weight} transformer.decoder.layers.3.norm1.{bias, weight} transformer.decoder.layers.3.norm2.{bias, weight} transformer.decoder.layers.3.norm3.{bias, weight} transformer.decoder.layers.3.self_attn.out_proj.{bias, weight} transformer.decoder.layers.3.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.4.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.4.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.4.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.4.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.4.linear1.{bias, weight} transformer.decoder.layers.4.linear2.{bias, weight} transformer.decoder.layers.4.norm1.{bias, weight} transformer.decoder.layers.4.norm2.{bias, weight} transformer.decoder.layers.4.norm3.{bias, weight} transformer.decoder.layers.4.self_attn.out_proj.{bias, weight} transformer.decoder.layers.4.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.5.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.5.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.5.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.5.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.5.linear1.{bias, weight} transformer.decoder.layers.5.linear2.{bias, weight} transformer.decoder.layers.5.norm1.{bias, weight} transformer.decoder.layers.5.norm2.{bias, weight} transformer.decoder.layers.5.norm3.{bias, weight} transformer.decoder.layers.5.self_attn.out_proj.{bias, weight} transformer.decoder.layers.5.self_attn.{in_proj_bias, in_proj_weight} transformer.encoder.layers.0.linear1.{bias, weight} transformer.encoder.layers.0.linear2.{bias, weight} transformer.encoder.layers.0.norm1.{bias, weight} transformer.encoder.layers.0.norm2.{bias, weight} transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.0.self_attn.output_proj.{bias, weight} transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.0.self_attn.value_proj.{bias, weight} transformer.encoder.layers.1.linear1.{bias, weight} transformer.encoder.layers.1.linear2.{bias, weight} transformer.encoder.layers.1.norm1.{bias, weight} transformer.encoder.layers.1.norm2.{bias, weight} transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.1.self_attn.output_proj.{bias, weight} transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.1.self_attn.value_proj.{bias, weight} transformer.encoder.layers.2.linear1.{bias, weight} transformer.encoder.layers.2.linear2.{bias, weight} transformer.encoder.layers.2.norm1.{bias, weight} transformer.encoder.layers.2.norm2.{bias, weight} transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.2.self_attn.output_proj.{bias, weight} transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.2.self_attn.value_proj.{bias, weight} transformer.encoder.layers.3.linear1.{bias, weight} transformer.encoder.layers.3.linear2.{bias, weight} transformer.encoder.layers.3.norm1.{bias, weight} transformer.encoder.layers.3.norm2.{bias, weight} transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.3.self_attn.output_proj.{bias, weight} transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.3.self_attn.value_proj.{bias, weight} transformer.encoder.layers.4.linear1.{bias, weight} transformer.encoder.layers.4.linear2.{bias, weight} transformer.encoder.layers.4.norm1.{bias, weight} transformer.encoder.layers.4.norm2.{bias, weight} transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.4.self_attn.output_proj.{bias, weight} transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.4.self_attn.value_proj.{bias, weight} transformer.encoder.layers.5.linear1.{bias, weight} transformer.encoder.layers.5.linear2.{bias, weight} transformer.encoder.layers.5.norm1.{bias, weight} transformer.encoder.layers.5.norm2.{bias, weight} transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.5.self_attn.output_proj.{bias, weight} transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.5.self_attn.value_proj.{bias, weight} transformer.level_embed. 是否是因为这样才导致训练的模型不好,在推理的时候没有结果?
WARNING [04/09 18:46:39 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: class_embed.0.{bias, weight} class_embed.1.{bias, weight} class_embed.2.{bias, weight} class_embed.3.{bias, weight} class_embed.4.{bias, weight} class_embed.5.{bias, weight} position.weight position_offset.weight query_embed.weight query_embed_offset.weight track_embed.linear1.{bias, weight} track_embed.linear2.{bias, weight} track_embed.linear_feat1.{bias, weight} track_embed.linear_feat2.{bias, weight} track_embed.norm1.{bias, weight} track_embed.norm2.{bias, weight} track_embed.norm_feat.{bias, weight} track_embed.self_attn.out_proj.{bias, weight} track_embed.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.class_embed.0.{bias, weight} transformer.decoder.class_embed.1.{bias, weight} transformer.decoder.class_embed.2.{bias, weight} transformer.decoder.class_embed.3.{bias, weight} transformer.decoder.class_embed.4.{bias, weight} transformer.decoder.class_embed.5.{bias, weight} transformer.decoder.layers.0.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.0.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.0.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.0.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.0.linear1.{bias, weight} transformer.decoder.layers.0.linear2.{bias, weight} transformer.decoder.layers.0.norm1.{bias, weight} transformer.decoder.layers.0.norm2.{bias, weight} transformer.decoder.layers.0.norm3.{bias, weight} transformer.decoder.layers.0.self_attn.out_proj.{bias, weight} transformer.decoder.layers.0.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.1.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.1.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.1.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.1.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.1.linear1.{bias, weight} transformer.decoder.layers.1.linear2.{bias, weight} transformer.decoder.layers.1.norm1.{bias, weight} transformer.decoder.layers.1.norm2.{bias, weight} transformer.decoder.layers.1.norm3.{bias, weight} transformer.decoder.layers.1.self_attn.out_proj.{bias, weight} transformer.decoder.layers.1.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.2.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.2.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.2.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.2.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.2.linear1.{bias, weight} transformer.decoder.layers.2.linear2.{bias, weight} transformer.decoder.layers.2.norm1.{bias, weight} transformer.decoder.layers.2.norm2.{bias, weight} transformer.decoder.layers.2.norm3.{bias, weight} transformer.decoder.layers.2.self_attn.out_proj.{bias, weight} transformer.decoder.layers.2.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.3.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.3.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.3.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.3.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.3.linear1.{bias, weight} transformer.decoder.layers.3.linear2.{bias, weight} transformer.decoder.layers.3.norm1.{bias, weight} transformer.decoder.layers.3.norm2.{bias, weight} transformer.decoder.layers.3.norm3.{bias, weight} transformer.decoder.layers.3.self_attn.out_proj.{bias, weight} transformer.decoder.layers.3.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.4.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.4.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.4.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.4.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.4.linear1.{bias, weight} transformer.decoder.layers.4.linear2.{bias, weight} transformer.decoder.layers.4.norm1.{bias, weight} transformer.decoder.layers.4.norm2.{bias, weight} transformer.decoder.layers.4.norm3.{bias, weight} transformer.decoder.layers.4.self_attn.out_proj.{bias, weight} transformer.decoder.layers.4.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.5.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.5.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.5.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.5.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.5.linear1.{bias, weight} transformer.decoder.layers.5.linear2.{bias, weight} transformer.decoder.layers.5.norm1.{bias, weight} transformer.decoder.layers.5.norm2.{bias, weight} transformer.decoder.layers.5.norm3.{bias, weight} transformer.decoder.layers.5.self_attn.out_proj.{bias, weight} transformer.decoder.layers.5.self_attn.{in_proj_bias, in_proj_weight} transformer.encoder.layers.0.linear1.{bias, weight} transformer.encoder.layers.0.linear2.{bias, weight} transformer.encoder.layers.0.norm1.{bias, weight} transformer.encoder.layers.0.norm2.{bias, weight} transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.0.self_attn.output_proj.{bias, weight} transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.0.self_attn.value_proj.{bias, weight} transformer.encoder.layers.1.linear1.{bias, weight} transformer.encoder.layers.1.linear2.{bias, weight} transformer.encoder.layers.1.norm1.{bias, weight} transformer.encoder.layers.1.norm2.{bias, weight} transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.1.self_attn.output_proj.{bias, weight} transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.1.self_attn.value_proj.{bias, weight} transformer.encoder.layers.2.linear1.{bias, weight} transformer.encoder.layers.2.linear2.{bias, weight} transformer.encoder.layers.2.norm1.{bias, weight} transformer.encoder.layers.2.norm2.{bias, weight} transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.2.self_attn.output_proj.{bias, weight} transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.2.self_attn.value_proj.{bias, weight} transformer.encoder.layers.3.linear1.{bias, weight} transformer.encoder.layers.3.linear2.{bias, weight} transformer.encoder.layers.3.norm1.{bias, weight} transformer.encoder.layers.3.norm2.{bias, weight} transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.3.self_attn.output_proj.{bias, weight} transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.3.self_attn.value_proj.{bias, weight} transformer.encoder.layers.4.linear1.{bias, weight} transformer.encoder.layers.4.linear2.{bias, weight} transformer.encoder.layers.4.norm1.{bias, weight} transformer.encoder.layers.4.norm2.{bias, weight} transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.4.self_attn.output_proj.{bias, weight} transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.4.self_attn.value_proj.{bias, weight} transformer.encoder.layers.5.linear1.{bias, weight} transformer.encoder.layers.5.linear2.{bias, weight} transformer.encoder.layers.5.norm1.{bias, weight} transformer.encoder.layers.5.norm2.{bias, weight} transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.5.self_attn.output_proj.{bias, weight} transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.5.self_attn.value_proj.{bias, weight} transformer.level_embed
您好!怎么在八张卡上面跑呢?通过命令行吗?我使用“./tools/simple_inference.sh configs/motrv2ch_uni5cost3ggoon.args ./pretrained_weight/checkpoint0019.pth”可以成功在DanceTrack test上测试,但是只用了一张卡
是的,代码测试只支持单卡。不过可以把代码改为分布式
谢谢您!我在train.sh里看到了支持四张卡训练的代码“python -m torch.distributed.launch --nproc_per_node=4 --master_port 29504 --use_env main.py ${args} --output_dir $OUTPUT_DIR |& tee -a output.log”,我去模仿一下,试试应用到代码测试部分!感谢您的工作以及回复
在使用detrex训练的时候,直接用那个代码会报错,错误及我的修改如下: TypeError: repr() got an unexpected keyword argument '_repr_indent' 我将之修改为: MS- deformable-attension报错 我将之改为: 形参问题: 我将之改为: 函数缺失 我将本仓库的merge函数加入进去: 至此,可以成功训练. 我使用的预训练参数是detrex中Deformable-DETR-R50 + Box-Refinement版本, 在加载的时候有很多不匹配的层: WARNING [04/09 18:46:39 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: class_embed.0.{bias, weight} class_embed.1.{bias, weight} class_embed.2.{bias, weight} class_embed.3.{bias, weight} class_embed.4.{bias, weight} class_embed.5.{bias, weight} position.weight position_offset.weight query_embed.weight query_embed_offset.weight track_embed.linear1.{bias, weight} track_embed.linear2.{bias, weight} track_embed.linear_feat1.{bias, weight} track_embed.linear_feat2.{bias, weight} track_embed.norm1.{bias, weight} track_embed.norm2.{bias, weight} track_embed.norm_feat.{bias, weight} track_embed.self_attn.out_proj.{bias, weight} track_embed.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.class_embed.0.{bias, weight} transformer.decoder.class_embed.1.{bias, weight} transformer.decoder.class_embed.2.{bias, weight} transformer.decoder.class_embed.3.{bias, weight} transformer.decoder.class_embed.4.{bias, weight} transformer.decoder.class_embed.5.{bias, weight} transformer.decoder.layers.0.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.0.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.0.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.0.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.0.linear1.{bias, weight} transformer.decoder.layers.0.linear2.{bias, weight} transformer.decoder.layers.0.norm1.{bias, weight} transformer.decoder.layers.0.norm2.{bias, weight} transformer.decoder.layers.0.norm3.{bias, weight} transformer.decoder.layers.0.self_attn.out_proj.{bias, weight} transformer.decoder.layers.0.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.1.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.1.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.1.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.1.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.1.linear1.{bias, weight} transformer.decoder.layers.1.linear2.{bias, weight} transformer.decoder.layers.1.norm1.{bias, weight} transformer.decoder.layers.1.norm2.{bias, weight} transformer.decoder.layers.1.norm3.{bias, weight} transformer.decoder.layers.1.self_attn.out_proj.{bias, weight} transformer.decoder.layers.1.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.2.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.2.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.2.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.2.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.2.linear1.{bias, weight} transformer.decoder.layers.2.linear2.{bias, weight} transformer.decoder.layers.2.norm1.{bias, weight} transformer.decoder.layers.2.norm2.{bias, weight} transformer.decoder.layers.2.norm3.{bias, weight} transformer.decoder.layers.2.self_attn.out_proj.{bias, weight} transformer.decoder.layers.2.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.3.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.3.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.3.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.3.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.3.linear1.{bias, weight} transformer.decoder.layers.3.linear2.{bias, weight} transformer.decoder.layers.3.norm1.{bias, weight} transformer.decoder.layers.3.norm2.{bias, weight} transformer.decoder.layers.3.norm3.{bias, weight} transformer.decoder.layers.3.self_attn.out_proj.{bias, weight} transformer.decoder.layers.3.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.4.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.4.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.4.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.4.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.4.linear1.{bias, weight} transformer.decoder.layers.4.linear2.{bias, weight} transformer.decoder.layers.4.norm1.{bias, weight} transformer.decoder.layers.4.norm2.{bias, weight} transformer.decoder.layers.4.norm3.{bias, weight} transformer.decoder.layers.4.self_attn.out_proj.{bias, weight} transformer.decoder.layers.4.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.5.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.5.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.5.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.5.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.5.linear1.{bias, weight} transformer.decoder.layers.5.linear2.{bias, weight} transformer.decoder.layers.5.norm1.{bias, weight} transformer.decoder.layers.5.norm2.{bias, weight} transformer.decoder.layers.5.norm3.{bias, weight} transformer.decoder.layers.5.self_attn.out_proj.{bias, weight} transformer.decoder.layers.5.self_attn.{in_proj_bias, in_proj_weight} transformer.encoder.layers.0.linear1.{bias, weight} transformer.encoder.layers.0.linear2.{bias, weight} transformer.encoder.layers.0.norm1.{bias, weight} transformer.encoder.layers.0.norm2.{bias, weight} transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.0.self_attn.output_proj.{bias, weight} transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.0.self_attn.value_proj.{bias, weight} transformer.encoder.layers.1.linear1.{bias, weight} transformer.encoder.layers.1.linear2.{bias, weight} transformer.encoder.layers.1.norm1.{bias, weight} transformer.encoder.layers.1.norm2.{bias, weight} transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.1.self_attn.output_proj.{bias, weight} transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.1.self_attn.value_proj.{bias, weight} transformer.encoder.layers.2.linear1.{bias, weight} transformer.encoder.layers.2.linear2.{bias, weight} transformer.encoder.layers.2.norm1.{bias, weight} transformer.encoder.layers.2.norm2.{bias, weight} transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.2.self_attn.output_proj.{bias, weight} transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.2.self_attn.value_proj.{bias, weight} transformer.encoder.layers.3.linear1.{bias, weight} transformer.encoder.layers.3.linear2.{bias, weight} transformer.encoder.layers.3.norm1.{bias, weight} transformer.encoder.layers.3.norm2.{bias, weight} transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.3.self_attn.output_proj.{bias, weight} transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.3.self_attn.value_proj.{bias, weight} transformer.encoder.layers.4.linear1.{bias, weight} transformer.encoder.layers.4.linear2.{bias, weight} transformer.encoder.layers.4.norm1.{bias, weight} transformer.encoder.layers.4.norm2.{bias, weight} transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.4.self_attn.output_proj.{bias, weight} transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.4.self_attn.value_proj.{bias, weight} transformer.encoder.layers.5.linear1.{bias, weight} transformer.encoder.layers.5.linear2.{bias, 58E9 weight} transformer.encoder.layers.5.norm1.{bias, weight} transformer.encoder.layers.5.norm2.{bias, weight} transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.5.self_attn.output_proj.{bias, weight} transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.5.self_attn.value_proj.{bias, weight} transformer.level_embed. 是否是因为这样才导致训练的模型不好,在推理的时候没有结果?
至此,可以成功训练. 我使用的预训练参数是detrex中Deformable-DETR-R50 + Box-Refinement版本, 在加载的时候有很多不匹配的层: WARNING [04/09 18:46:39 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: class_embed.0.{bias, weight} class_embed.1.{bias, weight} class_embed.2.{bias, weight} class_embed.3.{bias, weight} class_embed.4.{bias, weight} class_embed.5.{bias, weight} position.weight position_offset.weight query_embed.weight query_embed_offset.weight track_embed.linear1.{bias, weight} track_embed.linear2.{bias, weight} track_embed.linear_feat1.{bias, weight} track_embed.linear_feat2.{bias, weight} track_embed.norm1.{bias, weight} track_embed.norm2.{bias, weight} track_embed.norm_feat.{bias, weight} track_embed.self_attn.out_proj.{bias, weight} track_embed.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.class_embed.0.{bias, weight} transformer.decoder.class_embed.1.{bias, weight} transformer.decoder.class_embed.2.{bias, weight} transformer.decoder.class_embed.3.{bias, weight} transformer.decoder.class_embed.4.{bias, weight} transformer.decoder.class_embed.5.{bias, weight} transformer.decoder.layers.0.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.0.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.0.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.0.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.0.linear1.{bias, weight} transformer.decoder.layers.0.linear2.{bias, weight} transformer.decoder.layers.0.norm1.{bias, weight} transformer.decoder.layers.0.norm2.{bias, weight} transformer.decoder.layers.0.norm3.{bias, weight} transformer.decoder.layers.0.self_attn.out_proj.{bias, weight} transformer.decoder.layers.0.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.1.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.1.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.1.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.1.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.1.linear1.{bias, weight} transformer.decoder.layers.1.linear2.{bias, weight} transformer.decoder.layers.1.norm1.{bias, weight} transformer.decoder.layers.1.norm2.{bias, weight} transformer.decoder.layers.1.norm3.{bias, weight} transformer.decoder.layers.1.self_attn.out_proj.{bias, weight} transformer.decoder.layers.1.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.2.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.2.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.2.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.2.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.2.linear1.{bias, weight} transformer.decoder.layers.2.linear2.{bias, weight} transformer.decoder.layers.2.norm1.{bias, weight} transformer.decoder.layers.2.norm2.{bias, weight} transformer.decoder.layers.2.norm3.{bias, weight} transformer.decoder.layers.2.self_attn.out_proj.{bias, weight} transformer.decoder.layers.2.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.3.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.3.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.3.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.3.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.3.linear1.{bias, weight} transformer.decoder.layers.3.linear2.{bias, weight} transformer.decoder.layers.3.norm1.{bias, weight} transformer.decoder.layers.3.norm2.{bias, weight} transformer.decoder.layers.3.norm3.{bias, weight} transformer.decoder.layers.3.self_attn.out_proj.{bias, weight} transformer.decoder.layers.3.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.4.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.4.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.4.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.4.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.4.linear1.{bias, weight} transformer.decoder.layers.4.linear2.{bias, weight} transformer.decoder.layers.4.norm1.{bias, weight} transformer.decoder.layers.4.norm2.{bias, weight} transformer.decoder.layers.4.norm3.{bias, weight} transformer.decoder.layers.4.self_attn.out_proj.{bias, weight} transformer.decoder.layers.4.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.5.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.5.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.5.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.5.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.5.linear1.{bias, weight} transformer.decoder.layers.5.linear2.{bias, weight} transformer.decoder.layers.5.norm1.{bias, weight} transformer.decoder.layers.5.norm2.{bias, weight} transformer.decoder.layers.5.norm3.{bias, weight} transformer.decoder.layers.5.self_attn.out_proj.{bias, weight} transformer.decoder.layers.5.self_attn.{in_proj_bias, in_proj_weight} transformer.encoder.layers.0.linear1.{bias, weight} transformer.encoder.layers.0.linear2.{bias, weight} transformer.encoder.layers.0.norm1.{bias, weight} transformer.encoder.layers.0.norm2.{bias, weight} transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.0.self_attn.output_proj.{bias, weight} transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.0.self_attn.value_proj.{bias, weight} transformer.encoder.layers.1.linear1.{bias, weight} transformer.encoder.layers.1.linear2.{bias, weight} transformer.encoder.layers.1.norm1.{bias, weight} transformer.encoder.layers.1.norm2.{bias, weight} transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.1.self_attn.output_proj.{bias, weight} transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.1.self_attn.value_proj.{bias, weight} transformer.encoder.layers.2.linear1.{bias, weight} transformer.encoder.layers.2.linear2.{bias, weight} transformer.encoder.layers.2.norm1.{bias, weight} transformer.encoder.layers.2.norm2.{bias, weight} transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.2.self_attn.output_proj.{bias, weight} transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.2.self_attn.value_proj.{bias, weight} transformer.encoder.layers.3.linear1.{bias, weight} transformer.encoder.layers.3.linear2.{bias, weight} transformer.encoder.layers.3.norm1.{bias, weight} transformer.encoder.layers.3.norm2.{bias, weight} transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.3.self_attn.output_proj.{bias, weight} transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.3.self_attn.value_proj.{bias, weight} transformer.encoder.layers.4.linear1.{bias, weight} transformer.encoder.layers.4.linear2.{bias, weight} transformer.encoder.layers.4.norm1.{bias, weight} transformer.encoder.layers.4.norm2.{bias, weight} transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.4.self_attn.output_proj.{bias, weight} transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.4.self_attn.value_proj.{bias, weight} transformer.encoder.layers.5.linear1.{bias, weight} transformer.encoder.layers.5.linear2.{bias, 58E9 weight} transformer.encoder.layers.5.norm1.{bias, weight} transformer.encoder.layers.5.norm2.{bias, weight} transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.5.self_attn.output_proj.{bias, weight} transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.5.self_attn.value_proj.{bias, weight} transformer.level_embed. 是否是因为这样才导致训练的模型不好,在推理的时候没有结果?
WARNING [04/09 18:46:39 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: class_embed.0.{bias, weight} class_embed.1.{bias, weight} class_embed.2.{bias, weight} class_embed.3.{bias, weight} class_embed.4.{bias, weight} class_embed.5.{bias, weight} position.weight position_offset.weight query_embed.weight query_embed_offset.weight track_embed.linear1.{bias, weight} track_embed.linear2.{bias, weight} track_embed.linear_feat1.{bias, weight} track_embed.linear_feat2.{bias, weight} track_embed.norm1.{bias, weight} track_embed.norm2.{bias, weight} track_embed.norm_feat.{bias, weight} track_embed.self_attn.out_proj.{bias, weight} track_embed.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.class_embed.0.{bias, weight} transformer.decoder.class_embed.1.{bias, weight} transformer.decoder.class_embed.2.{bias, weight} transformer.decoder.class_embed.3.{bias, weight} transformer.decoder.class_embed.4.{bias, weight} transformer.decoder.class_embed.5.{bias, weight} transformer.decoder.layers.0.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.0.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.0.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.0.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.0.linear1.{bias, weight} transformer.decoder.layers.0.linear2.{bias, weight} transformer.decoder.layers.0.norm1.{bias, weight} transformer.decoder.layers.0.norm2.{bias, weight} transformer.decoder.layers.0.norm3.{bias, weight} transformer.decoder.layers.0.self_attn.out_proj.{bias, weight} transformer.decoder.layers.0.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.1.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.1.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.1.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.1.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.1.linear1.{bias, weight} transformer.decoder.layers.1.linear2.{bias, weight} transformer.decoder.layers.1.norm1.{bias, weight} transformer.decoder.layers.1.norm2.{bias, weight} transformer.decoder.layers.1.norm3.{bias, weight} transformer.decoder.layers.1.self_attn.out_proj.{bias, weight} transformer.decoder.layers.1.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.2.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.2.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.2.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.2.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.2.linear1.{bias, weight} transformer.decoder.layers.2.linear2.{bias, weight} transformer.decoder.layers.2.norm1.{bias, weight} transformer.decoder.layers.2.norm2.{bias, weight} transformer.decoder.layers.2.norm3.{bias, weight} transformer.decoder.layers.2.self_attn.out_proj.{bias, weight} transformer.decoder.layers.2.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.3.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.3.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.3.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.3.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.3.linear1.{bias, weight} transformer.decoder.layers.3.linear2.{bias, weight} transformer.decoder.layers.3.norm1.{bias, weight} transformer.decoder.layers.3.norm2.{bias, weight} transformer.decoder.layers.3.norm3.{bias, weight} transformer.decoder.layers.3.self_attn.out_proj.{bias, weight} transformer.decoder.layers.3.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.4.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.4.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.4.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.4.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.4.linear1.{bias, weight} transformer.decoder.layers.4.linear2.{bias, weight} transformer.decoder.layers.4.norm1.{bias, weight} transformer.decoder.layers.4.norm2.{bias, weight} transformer.decoder.layers.4.norm3.{bias, weight} transformer.decoder.layers.4.self_attn.out_proj.{bias, weight} transformer.decoder.layers.4.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.5.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.5.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.5.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.5.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.5.linear1.{bias, weight} transformer.decoder.layers.5.linear2.{bias, weight} transformer.decoder.layers.5.norm1.{bias, weight} transformer.decoder.layers.5.norm2.{bias, weight} transformer.decoder.layers.5.norm3.{bias, weight} transformer.decoder.layers.5.self_attn.out_proj.{bias, weight} transformer.decoder.layers.5.self_attn.{in_proj_bias, in_proj_weight} transformer.encoder.layers.0.linear1.{bias, weight} transformer.encoder.layers.0.linear2.{bias, weight} transformer.encoder.layers.0.norm1.{bias, weight} transformer.encoder.layers.0.norm2.{bias, weight} transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.0.self_attn.output_proj.{bias, weight} transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.0.self_attn.value_proj.{bias, weight} transformer.encoder.layers.1.linear1.{bias, weight} transformer.encoder.layers.1.linear2.{bias, weight} transformer.encoder.layers.1.norm1.{bias, weight} transformer.encoder.layers.1.norm2.{bias, weight} transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.1.self_attn.output_proj.{bias, weight} transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.1.self_attn.value_proj.{bias, weight} transformer.encoder.layers.2.linear1.{bias, weight} transformer.encoder.layers.2.linear2.{bias, weight} transformer.encoder.layers.2.norm1.{bias, weight} transformer.encoder.layers.2.norm2.{bias, weight} transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.2.self_attn.output_proj.{bias, weight} transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.2.self_attn.value_proj.{bias, weight} transformer.encoder.layers.3.linear1.{bias, weight} transformer.encoder.layers.3.linear2.{bias, weight} transformer.encoder.layers.3.norm1.{bias, weight} transformer.encoder.layers.3.norm2.{bias, weight} transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.3.self_attn.output_proj.{bias, weight} transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.3.self_attn.value_proj.{bias, weight} transformer.encoder.layers.4.linear1.{bias, weight} transformer.encoder.layers.4.linear2.{bias, weight} transformer.encoder.layers.4.norm1.{bias, weight} transformer.encoder.layers.4.norm2.{bias, weight} transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.4.self_attn.output_proj.{bias, weight} transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.4.self_attn.value_proj.{bias, weight} transformer.encoder.layers.5.linear1.{bias, weight} transformer.encoder.layers.5.linear2.{bias, 58E9 weight} transformer.encoder.layers.5.norm1.{bias, weight} transformer.encoder.layers.5.norm2.{bias, weight} transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.5.self_attn.output_proj.{bias, weight} transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.5.self_attn.value_proj.{bias, weight} transformer.level_embed
您好!第四步缺失的merge函数,在哪里能找到呢?
感觉是[Deformable-DETR-R50 + Box-Refinement]与comot实用的backbone不是同一个。
Activity
fengxiuyaun commentedon Apr 8, 2024
我用v100是训一个epoch大概11个小时,第一个epoch会慢一些
WesLee88524 commentedon Apr 8, 2024
感谢!
请问使用detrex版本的CO-MOT速度会快点吗? 我看那个batch_size可以自己设定,是否可以公布训练代码?我使用那个版本的代码重新训练后测试无结果.
fengxiuyaun commentedon Apr 8, 2024
那个我只是简单写了batch_size>1的代码,简单测试了一下没啥大问题,就递交了。不过你可以参考看看有啥我没发现的问题。
WesLee88524 commentedon Apr 8, 2024
好嘞! 感谢!我再研究下!
WesLee88524 commentedon Apr 8, 2024
您好,我全部对照了这个版本的CO-MOT和detrex的CO-MOT代码,感觉是什么问题的,训练的过程好像也对,loss都是正常的,但是训练出来的模型评估结果一直为空,请问可能是什么原因呢?我的配置文件如下:

训练的命令为:

评估结果如下:

fengxiuyaun commentedon Apr 9, 2024
为空啥意思,就是没有识别到?
fengxiuyaun commentedon Apr 9, 2024
你可以先跑一下我release的权重,看看代码有啥问题
WesLee88524 commentedon Apr 9, 2024
对的, 我按照这个配置文件重新训练然后评估,输出文件每帧都是空的,如下图:

, 我用您release的权重直接测试,HOTA在dancetrack val上比论文低4.2%,但是可以跑通. 之后我又对比了各个主要文件,好像都没啥问题. 但训练就是不行😂
WesLee88524 commentedon Apr 9, 2024
在使用detrex训练的时候,直接用那个代码会报错,错误及我的修改如下:
TypeError: repr() got an unexpected keyword argument '_repr_indent'


我将之修改为:
MS- deformable-attension报错


我将之改为:
形参问题:


我将之改为:
函数缺失


我将本仓库的merge函数加入进去:
至此,可以成功训练.
我使用的预训练参数是detrex中Deformable-DETR-R50 + Box-Refinement版本, 在加载的时候有很多不匹配的层:
WARNING [04/09 18:46:39 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: class_embed.0.{bias, weight} class_embed.1.{bias, weight} class_embed.2.{bias, weight} class_embed.3.{bias, weight} class_embed.4.{bias, weight} class_embed.5.{bias, weight} position.weight position_offset.weight query_embed.weight query_embed_offset.weight track_embed.linear1.{bias, weight} track_embed.linear2.{bias, weight} track_embed.linear_feat1.{bias, weight} track_embed.linear_feat2.{bias, weight} track_embed.norm1.{bias, weight} track_embed.norm2.{bias, weight} track_embed.norm_feat.{bias, weight} track_embed.self_attn.out_proj.{bias, weight} track_embed.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.class_embed.0.{bias, weight} transformer.decoder.class_embed.1.{bias, weight} transformer.decoder.class_embed.2.{bias, weight} transformer.decoder.class_embed.3.{bias, weight} transformer.decoder.class_embed.4.{bias, weight} transformer.decoder.class_embed.5.{bias, weight} transformer.decoder.layers.0.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.0.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.0.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.0.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.0.linear1.{bias, weight} transformer.decoder.layers.0.linear2.{bias, weight} transformer.decoder.layers.0.norm1.{bias, weight} transformer.decoder.layers.0.norm2.{bias, weight} transformer.decoder.layers.0.norm3.{bias, weight} transformer.decoder.layers.0.self_attn.out_proj.{bias, weight} transformer.decoder.layers.0.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.1.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.1.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.1.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.1.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.1.linear1.{bias, weight} transformer.decoder.layers.1.linear2.{bias, weight} transformer.decoder.layers.1.norm1.{bias, weight} transformer.decoder.layers.1.norm2.{bias, weight} transformer.decoder.layers.1.norm3.{bias, weight} transformer.decoder.layers.1.self_attn.out_proj.{bias, weight} transformer.decoder.layers.1.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.2.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.2.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.2.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.2.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.2.linear1.{bias, weight} transformer.decoder.layers.2.linear2.{bias, weight} transformer.decoder.layers.2.norm1.{bias, weight} transformer.decoder.layers.2.norm2.{bias, weight} transformer.decoder.layers.2.norm3.{bias, weight} transformer.decoder.layers.2.self_attn.out_proj.{bias, weight} transformer.decoder.layers.2.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.3.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.3.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.3.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.3.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.3.linear1.{bias, weight} transformer.decoder.layers.3.linear2.{bias, weight} transformer.decoder.layers.3.norm1.{bias, weight} transformer.decoder.layers.3.norm2.{bias, weight} transformer.decoder.layers.3.norm3.{bias, weight} transformer.decoder.layers.3.self_attn.out_proj.{bias, weight} transformer.decoder.layers.3.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.4.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.4.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.4.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.4.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.4.linear1.{bias, weight} transformer.decoder.layers.4.linear2.{bias, weight} transformer.decoder.layers.4.norm1.{bias, weight} transformer.decoder.layers.4.norm2.{bias, weight} transformer.decoder.layers.4.norm3.{bias, weight} transformer.decoder.layers.4.self_attn.out_proj.{bias, weight} transformer.decoder.layers.4.self_attn.{in_proj_bias, in_proj_weight} transformer.decoder.layers.5.cross_attn.attention_weights.{bias, weight} transformer.decoder.layers.5.cross_attn.output_proj.{bias, weight} transformer.decoder.layers.5.cross_attn.sampling_offsets.{bias, weight} transformer.decoder.layers.5.cross_attn.value_proj.{bias, weight} transformer.decoder.layers.5.linear1.{bias, weight} transformer.decoder.layers.5.linear2.{bias, weight} transformer.decoder.layers.5.norm1.{bias, weight} transformer.decoder.layers.5.norm2.{bias, weight} transformer.decoder.layers.5.norm3.{bias, weight} transformer.decoder.layers.5.self_attn.out_proj.{bias, weight} transformer.decoder.layers.5.self_attn.{in_proj_bias, in_proj_weight} transformer.encoder.layers.0.linear1.{bias, weight} transformer.encoder.layers.0.linear2.{bias, weight} transformer.encoder.layers.0.norm1.{bias, weight} transformer.encoder.layers.0.norm2.{bias, weight} transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.0.self_attn.output_proj.{bias, weight} transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.0.self_attn.value_proj.{bias, weight} transformer.encoder.layers.1.linear1.{bias, weight} transformer.encoder.layers.1.linear2.{bias, weight} transformer.encoder.layers.1.norm1.{bias, weight} transformer.encoder.layers.1.norm2.{bias, weight} transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.1.self_attn.output_proj.{bias, weight} transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.1.self_attn.value_proj.{bias, weight} transformer.encoder.layers.2.linear1.{bias, weight} transformer.encoder.layers.2.linear2.{bias, weight} transformer.encoder.layers.2.norm1.{bias, weight} transformer.encoder.layers.2.norm2.{bias, weight} transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.2.self_attn.output_proj.{bias, weight} transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.2.self_attn.value_proj.{bias, weight} transformer.encoder.layers.3.linear1.{bias, weight} transformer.encoder.layers.3.linear2.{bias, weight} transformer.encoder.layers.3.norm1.{bias, weight} transformer.encoder.layers.3.norm2.{bias, weight} transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.3.self_attn.output_proj.{bias, weight} transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.3.self_attn.value_proj.{bias, weight} transformer.encoder.layers.4.linear1.{bias, weight} transformer.encoder.layers.4.linear2.{bias, weight} transformer.encoder.layers.4.norm1.{bias, weight} transformer.encoder.layers.4.norm2.{bias, weight} transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.4.self_attn.output_proj.{bias, weight} transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.4.self_attn.value_proj.{bias, weight} transformer.encoder.layers.5.linear1.{bias, weight} transformer.encoder.layers.5.linear2.{bias, weight} transformer.encoder.layers.5.norm1.{bias, weight} transformer.encoder.layers.5.norm2.{bias, weight} transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight} transformer.encoder.layers.5.self_attn.output_proj.{bias, weight} transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight} transformer.encoder.layers.5.self_attn.value_proj.{bias, weight} transformer.level_embed.是否是因为这样才导致训练的模型不好,在推理的时候没有结果?
fengxiuyaun commentedon Apr 10, 2024
65.925 77.589 56.232 83.03 89.167 60.291 81.243 90.792 68.317 74.529 88.032 65.609 87.115 89.706 87.696 90.406 97.089 75.092 23.077 1.8315 203548 21600 6103 1308 205 63 5 3712 77.808 68.824 66.456 71.368 149624 75524 60027 209651 225148 671 273
我建议还是先对齐测试,通过测试来到环境或者代码有啥问题。假如方便可以加我WX:bphengyan2,这样方便沟通
fufuforu commentedon Mar 18, 2025
您好!怎么在八张卡上面跑呢?通过命令行吗?我使用“./tools/simple_inference.sh configs/motrv2ch_uni5cost3ggoon.args ./pretrained_weight/checkpoint0019.pth”可以成功在DanceTrack test上测试,但是只用了一张卡
fengxiuyaun commentedon Mar 18, 2025
是的,代码测试只支持单卡。不过可以把代码改为分布式
fufuforu commentedon Mar 18, 2025
谢谢您!我在train.sh里看到了支持四张卡训练的代码“python -m torch.distributed.launch --nproc_per_node=4 --master_port 29504 --use_env main.py ${args} --output_dir $OUTPUT_DIR |& tee -a output.log”,我去模仿一下,试试应用到代码测试部分!感谢您的工作以及回复
fufuforu commentedon Apr 8, 2025
您好!第四步缺失的merge函数,在哪里能找到呢?
fengxiuyaun commentedon Apr 8, 2025
感觉是[Deformable-DETR-R50 + Box-Refinement]与comot实用的backbone不是同一个。