token_count: 9 char_count: 44 digit_count: 6 alpha_count: 32 has_name: False numbers_found: [52, 2020, 21] num_count: 3 num_sum: 2093 num_avg: 697.666... email_domains_mentioned: ['yahoo', 'gmail', 'mail'] email_domain_count: 3 possible_emails: [] years_found: [2020] file_extension: txt looks_like_filename: True bigrams: ['stephen 52', '52 yahoo', 'yahoo com', 'com gmail', 'gmail com', 'com mail', 'mail com', 'com 2020', '2020 21', '21 txt'] year_num_pair: (2020, 21) entropy: 3.892 from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') embedding = model.encode(raw) features['sentence_embedding'] = embedding # 384-dim vector If by “make a deep feature” you meant something else (e.g., a neural net feature map, a regex to extract a password/username, or a data pipeline), let me know and I’ll adjust.
return features features = extract_deep_features("stephen 52 yahoo com gmail com mail com 2020 21 txt") Step 3 – Output the deep features for k, v in features.items(): print(f"{k}: {v}") Output example: stephen 52 yahoo com gmail com mail com 2020 21 txt
"stephen 52 yahoo com gmail com mail com 2020 21 txt" A deep feature in machine learning or data processing typically means extracting meaningful, higher-level attributes from raw input — going beyond simple keyword extraction into inferred patterns, relationships, or embeddings. token_count: 9 char_count: 44 digit_count: 6 alpha_count: 32
# 10. Text entropy (as a measure of unpredictability) import math freq = {} for ch in text: freq[ch] = freq.get(ch, 0) + 1 entropy = -sum((count/len(text)) * math.log2(count/len(text)) for count in freq.values()) features['entropy'] = round(entropy, 3) '21 txt'] year_num_pair: (2020
# 8. Pairwise patterns (bigrams) bigrams = [' '.join(tokens[i:i+2]) for i in range(len(tokens)-1)] features['bigrams'] = bigrams