input_texts = [] target_texts = [] input_characters = set() target_characters = set() with open(data_path, 'r', encoding='utf-8') as f: lines = f.read().split('\n') if len(lines) < num_samples: num_samples = len(lines) - 1 for line in lines[:num_samples]: try: input_text, target_text = line.split('\t') target_text = '\t' + target_text + '\n' input_texts.append(input_text) target_texts.append(target_text) for char in input_text.strip(): input_characters.add(char) for char in target_text.strip(): target_characters.add(char) except ValueError: pass input_characters = sorted(list(input_characters)) target_characters = sorted(list(target_characters)) num_encoder_tokens = len(input_characters) num_decoder_tokens = len(target_characters) max_encoder_seq_length = max([len(txt.strip()) for txt in input_texts]) max_decoder_seq_length = max([len(txt.strip()) for txt in target_texts])在上面代码加一个条件判断,判断input text是不是空,如果是空就返回其他,如果不是空再返回那个max等式
时间: 2024-02-25 17:57:48 浏览: 70
find_file_by_inline_texts
可以按照以下方式添加条件判断:
```
input_texts = []
target_texts = []
input_characters = set()
target_characters = set()
with open(data_path, 'r', encoding='utf-8') as f:
lines = f.read().split('\n')
if len(lines) < num_samples:
num_samples = len(lines) - 1
for line in lines[:num_samples]:
try:
input_text, target_text = line.split('\t')
target_text = '\t' + target_text + '\n'
if input_text.strip() == "":
continue # 如果input text是空,跳过本次循环
input_texts.append(input_text)
target_texts.append(target_text)
for char in input_text.strip():
input_characters.add(char)
for char in target_text.strip():
target_characters.add(char)
except ValueError:
pass
if not input_texts:
print("No valid input texts found.")
else:
input_characters = sorted(list(input_characters))
target_characters = sorted(list(target_characters))
num_encoder_tokens = len(input_characters)
num_decoder_tokens = len(target_characters)
max_encoder_seq_length = max([len(txt.strip()) for txt in input_texts])
max_decoder_seq_length = max([len(txt.strip()) for txt in target_texts])
```
这样,在 input_texts 中没有有效的 input text 时,就会输出 "No valid input texts found." 的提示。
阅读全文