8000 Trying to fine tune with different data, But getting dimensionality mismatched for tensor · Issue #18 · localminimum/QANet · GitHub
[go: up one dir, main page]

Skip to content
  • Trying to fine tune with different data, But getting dimensionality mismatched for tensor #18

    Open

    Description

    @brojokm

    I am getting the following error while trying to fine tune

    InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [326,64] rhs shape= [1427,64]
    	 [[Node: save/Assign_746 = Assign[T=DT_FLOAT, _class=["loc:@char_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_mat, save/RestoreV2:746)]]
    

    Activity

    1. localminimum commented on May 2, 2018

      @localminimum
      Owner

      Hi,
      Try re running the config.py --mode prepro if you make changes regarding embedding matrices

    2. brojokm commented on May 2, 2018

      @brojokm
      Author

      @localminimum
      If I change training file then after running preprocessing, embedding file will change and its dimension is mismatched with the existing one. That is what I have noticed as per my knowledge.
      Note: dimension does not mean embedding vector size of char/word.. Basically its giving error because number of words and chars are different. You can see here lhs shape= [326,64] rhs shape= [1427,64]

      kindly help me to resolve this.

    3. localminimum commented on May 3, 2018

      @localminimum
      Owner

      It's not that the number of words and chars are different. tf.assign happens when the pretrained weights are loaded.

      It seems like you are trying to load a trained model where the character embeddings size is different to your current character embeddings size. Try training again from scratch with your new char embeddings.

    4. brojokm commented on May 3, 2018

      @brojokm
      Author
    5. localminimum commented on May 3, 2018

      @localminimum
      Owner

      You can just train the new character embeddings from scratch while using the pretrained weights for other weights. But I don't have that feature implemented here. You might have to manually allocate each weight.

    6. brojokm commented on May 3, 2018

      @brojokm
      Author

      Hi! I'm new to tensorflow, and while I understand that I need to manually allocate weights to layers for the non-character embedding layers (it seems that word embeddings are non-trainable). Also, I don't understand how to restore only selected portions from checkpoint. It'll be really helpful if you could elaborate on what I need to do (or sample code)
      Thanks in advance :)

    7. anshulsolanki commented on Nov 15, 2018

      @anshulsolanki
      47A5

      @brojokm and @localminimum were you able to resolve this.
      I am getting the same error.
      Any help on how to "just train the new character embeddings from scratch while using the pretrained weights for other weights" is appreciated.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      No labels
      No labels

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions

      • 0