8000 Trying to fine tune with different data, But getting dimensionality mismatched for tensor · Issue #18 · localminimum/QANet · GitHub
[go: up one dir, main page]

Skip to content

Trying to fine tune with different data, But getting dimensionality mismatched for tensor #18

Open

Description

@brojokm

I am getting the following error while trying to fine tune

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [326,64] rhs shape= [1427,64]
	 [[Node: save/Assign_746 = Assign[T=DT_FLOAT, _class=["loc:@char_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_mat, save/RestoreV2:746)]]

Activity

  1. localminimum commented on May 2, 2018

    @localminimum
    Owner

    Hi,
    Try re running the config.py --mode prepro if you make changes regarding embedding matrices

  2. brojokm commented on May 2, 2018

    @brojokm
    Author

    @localminimum
    If I change training file then after running preprocessing, embedding file will change and its dimension is mismatched with the existing one. That is what I have noticed as per my knowledge.
    Note: dimension does not mean embedding vector size of char/word.. Basically its giving error because number of words and chars are different. You can see here lhs shape= [326,64] rhs shape= [1427,64]

    kindly help me to resolve this.

  3. localminimum commented on May 3, 2018

    @localminimum
    Owner

    It's not that the number of words and chars are different. tf.assign happens when the pretrained weights are loaded.

    It seems like you are trying to load a trained model where the character embeddings size is different to your current character embeddings size. Try training again from scratch with your new char embeddings.

  4. brojokm commented on May 3, 2018

    @brojokm
    Author
  5. localminimum commented on May 3, 2018

    @localminimum
    Owner

    You can just train the new character embeddings from scratch while using the pretrained weights for other weights. But I don't have that feature implemented here. You might have to manually allocate each weight.

  6. brojokm commented on May 3, 2018

    @brojokm
    Author

    Hi! I'm new to tensorflow, and while I understand that I need to manually allocate weights to layers for the non-character embedding layers (it seems that word embeddings are non-trainable). Also, I don't understand how to restore only selected portions from checkpoint. It'll be really helpful if you could elaborate on what I need to do (or sample code)
    Thanks in advance :)

  7. anshulsolanki commented on Nov 15, 2018

    @anshulsolanki

    @brojokm and @localminimum were you able to resolve this.
    I am getting the same error.
    Any help on how to "just train the new character embeddings from scratch while using the pretrained weights for other weights" is appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned
    3BDB

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0