Models

bert

函数式融合编程的BERT模型实现，对标huggingface

class mindnlp.models.bert.BertAttention(config)

基类：Cell

Bert Attention

construct(input_tensor, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertConfig(vocab_size=30522, hidden_size=768, num_hidden_layers=12, num_attention_heads=12, intermediate_size=3072, hidden_act='gelu', hidden_dropout_prob=0.1, attention_probs_dropout_prob=0.1, max_position_embeddings=512, type_vocab_size=2, initializer_range=0.02, layer_norm_eps=1e-12, **kwargs)

基类：PretrainedConfig

BERT-base配置类

class mindnlp.models.bert.BertEmbeddings(config)

基类：Cell

Bert编码层，包括word embeddings，position embeddings和token type embeddings

construct(input_ids, token_type_ids=None, position_ids=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertEncoder(config)

基类：Cell

Bert Encoder

construct(hidden_states, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertForPretraining(config, *args, **kwargs)

基类：Cell

Bert For Pretraining

construct(input_ids, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, masked_lm_positions=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertIntermediate(config)

基类：Cell

Bert Intermediate

construct(hidden_states)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertLMPredictionHead(config)

基类：Cell

Bert LM Prediction Head

construct(hidden_states, masked_lm_positions)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertLayer(config)

基类：Cell

Bert Layer

construct(hidden_states, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertModel(config)

基类：Cell

Bert Model

construct(input_ids, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertOutput(config)

基类：Cell

Bert Output

construct(hidden_states, input_tensor)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertPooler(config)

基类：Cell

Bert Pooler

construct(hidden_states)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertPreTrainingHeads(config)

基类：Cell

Bert PreTraining Heads

construct(sequence_output, pooled_output, masked_lm_positions)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertPredictionHeadTransform(config)

基类：Cell

Bert Prediction Head Transform

construct(hidden_states)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.BertSelfAttention(config)

基类：Cell

Self attention layer for BERT.

construct(hidden_states, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

transpose_for_scores(input_x)

transpose for scores

class mindnlp.models.bert.BertSelfOutput(config)

基类：Cell

Bert Self Output

construct(hidden_states, input_tensor)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.Matmul(auto_prefix=True, flags=None)

基类：Cell

Matmul Operation

construct(a, b)

Defines the computation to be performed. This method must be overridden by all subclasses.

备注

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

参数

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

返回

Tensor, returns the computed result.

class mindnlp.models.bert.PretrainedConfig(**kwargs)

基类：object

Pretrained Config.