algerianDeBERTa-realestate-ner

A Name Entity Recognition model based on AlgerianDeBERTa, finetuned to extract 13 real-estate entities from Algerian Facebook posts

Handles the exact language mix found on Facebook Marketplace and Algerian classified groups: Darja (Algerian dialect), Arabizi (Arabic written in Latin script), French, and heavy code-switching, finetuned on +7k examples of pure algerian real-estate posts from Facebook

---

Model Highlights

Architecture	DeBERTa-v2 — 12 layers, hidden=512, 8 heads, 2048 FFN
Base model	algerianDeBERTa (pre-trained on Algerian web text)
Task	Token classification — 27 BIO labels, 13 entity types
Languages	Algerian Darja · Arabizi · French · MSA · Code-switched
Domain	Real estate classifieds (sales, rentals, land, villas, apartments)
Test F1	0.9672 micro (seqeval, strict entity-level)
Best val F1	0.9858
Parameters	~60M
License	Apache 2.0

---

Quick Start

Option 1 : pipeline (standard, recommended for short texts)

Uses aggregation_strategy="max": for each surface word the subword token with the highest entity-class score wins, then consecutive spans that have the same type are merged automatically

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="81melody/algerianDeBERTa-realestate-ner",
    aggregation_strategy="max",
)


print(ner("سلام ، خصني اف2 فالعاصمة في ميسوني ولا اودان ولا ديدوش ، في هاد الجويه لي عندو يتوصل معيا في الخاص"))

print(ner("Appartement F4 à vendre Oran centre 120m² 4ème étage acte notarié"))

For texts that may exceed 192 tokens, pass sliding-window arguments directly to the pipeline call:

result = ner(
    long_text,
    truncation=True,
    max_length=192,
    stride=64,
)

Each result dict contains entity_group, word, score, start, end.

---

Option 2 : Manual sliding window inference (production / long texts)

For real estate posts that frequently exceed one chunk + to adapt with The small vocab size of the first version of the base model (30k), the approach below is more robust for production use: it averages the probability vectors of overlapping tokens across all chunks, then merges subword pieces that form the same surface word before BIO decoding

This fixes a common artefact where words like "cherche" (tokenised as ["cher", "che"]) or prices like "1.700" (tokenised as ["1", ".", "700"]) get truncated mid-word if a trailing subword happens to predict O

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
import numpy as np
from typing import List

MODEL_NAME  = "81melody/algerianDeBERTa-realestate-ner"
MAX_SEQ_LEN = 192
STRIDE      = 64

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model     = AutoModelForTokenClassification.from_pretrained(MODEL_NAME)
model.eval()
device    = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)


def extract_entities(text: str) -> List[dict]:
   
    enc = tokenizer(
        text,
        return_tensors="pt",
        max_length=MAX_SEQ_LEN,
        stride=STRIDE,
        truncation=True,
        return_overflowing_tokens=True,
        return_offsets_mapping=True,
        padding="max_length",
    )
    enc.pop("overflow_to_sample_mapping", None)
    offsets = enc.pop("offset_mapping")          


    with torch.no_grad():
        logits = model(**{k: v.to(device) for k, v in enc.items()}).logits
    probs     = torch.softmax(logits, dim=-1).cpu().numpy()  
    attention = enc["attention_mask"].numpy()                
    off_np    = offsets.numpy()                             


    char_probs = {}
    for c in range(probs.shape[0]):
        for t in range(off_np.shape[1]):
            if attention[c, t] == 0:
                continue
            cs, ce = int(off_np[c, t, 0]), int(off_np[c, t, 1])
            if cs == 0 and ce == 0:         
                continue
            if cs not in char_probs:
                char_probs[cs] = {"end": ce, "vecs": [probs[c, t]]}
            else:
                char_probs[cs]["vecs"].append(probs[c, t])

    if not char_probs:
        return []

    items = sorted(char_probs.items())

    word_tokens = []
    w_start, w_info = items[0]
    w_end   = w_info["end"]
    w_avg_p = np.mean(w_info["vecs"], axis=0)  

    for cs, info in items[1:]:
        if cs == w_end:                         
            w_end = info["end"]
        else:                                    
            word_tokens.append({"start": w_start, "end": w_end, "avg_p": w_avg_p})
            w_start = cs
            w_end   = info["end"]
            w_avg_p = np.mean(info["vecs"], axis=0)
    word_tokens.append({"start": w_start, "end": w_end, "avg_p": w_avg_p})

    label_map = model.config.id2label
    entities, current = [], None

    for w in word_tokens:
        idx   = int(np.argmax(w["avg_p"]))
        label = label_map[idx]
        score = float(w["avg_p"][idx])

        if label == "O":
            if current:
                entities.append(current)
                current = None

        elif label.startswith("B-"):
            if current:
                entities.append(current)
            current = {
                "entity": label[2:],
                "word":   text[w["start"]:w["end"]],
                "score":  score,
                "_sc": [score], "_s": w["start"], "_e": w["end"],
            }

        elif label.startswith("I-"):
            etype = label[2:]
            if current and current["entity"] == etype:
                current["word"]  = text[current["_s"]:w["end"]]
                current["_e"]    = w["end"]
                current["_sc"].append(score)
                current["score"] = float(np.mean(current["_sc"]))
            else:
                if current:
                    entities.append(current)
                current = {
                    "entity": etype,
                    "word":   text[w["start"]:w["end"]],
                    "score":  score,
                    "_sc": [score], "_s": w["start"], "_e": w["end"],
                }

    if current:
        entities.append(current)

    return [
        {"entity": e["entity"], "word": e["word"], "score": round(e["score"], 6)}
        for e in entities
    ]

---

Entity Schema

The model uses a 27-label BIO scheme covering 13 entity types drawn directly from Algerian real estate Facebook market

Entity	Description	Algerian Examples
PROPERTY_TYPE	Category of asset	شقة · villa · appartement · terrain · carcasse · haouch
APT_CLASS	Apartment layout	F2 · F3 · F4 · F5 · R+1 · Studio
TRANSACTION	Listing intent	للبيع · location · louer · خاصني · echange
WILAYA	Algerian province (all 48)	Alger · Oran · Constantine · 16 · ولاية وهران
CITY	City / commune	Bab Ezzouar · Sidi Yahia · Ain Benian
NEIGHBORHOOD	Quarter / district / street	باب الزوار · Hydra · Hai Yasmine · Télemly
PRICE	Price in DZD, DA, or slang	8 500 000 da · 12M · 950 000 · 1.5 milliards
SURFACE	Area in m², metres, hectares	90m² · 120 mètres · 85 متر · 2 hectares
FLOOR	Floor level	3ème étage · الطابق الثالث · RDC · R+2
PHONE	Contact number (anonymized)	[PHONE]
AMENITY	Features / utilities	garage · مصعد · piscine · jardin · بيدون
DOCUMENT	Legal papers	عقد · livret foncier · AADL · acte notarié · timbre
CONDITION	Property state	neuf · rénové · قديم · semi-fini · en construction

---

Performance

Evaluated on a held-out test set of 95 posts / 596 tokenized chunks — never seen during training or validation.

               precision    recall  f1-score   support

      AMENITY     0.9735    0.9971    0.9852       700
    APT_CLASS     0.9728    1.0000    0.9862       357
    CONDITION     0.9793    0.9861    0.9827       144
     DOCUMENT     0.9755    0.9848    0.9801       526
        FLOOR     0.9416    0.9928    0.9665       276
 NEIGHBORHOOD     0.8434    0.8642    0.8537        81
        PHONE     0.8101    0.9846    0.8889        65
        PRICE     0.8739    0.9336    0.9028       557
PROPERTY_TYPE     0.9869    0.9912    0.9890       682
      SURFACE     0.9259    0.9420    0.9338       517
  TRANSACTION     0.9950    0.9934    0.9942       603
       WILAYA     0.9810    0.9841    0.9825       629

    micro avg     0.9566    0.9780    0.9672      5137
    macro avg     0.9382    0.9712    0.9538      5137
 weighted avg     0.9576    0.9780    0.9675      5137

Metric: seqeval entity-level strict match (not token-level an entity prediction counts as correct only if both the span and the label match the annotation exactly)

---

Training Data

Sourced from the companion dataset 81melody/algerian-realestate-ner-dataset.

Split	Raw posts	Tokenized chunks
Train	1,316	7,138
Val	87	586
Test	95	596
Total	1,498	8,320

Language distribution: Arabic (Darja) 53% · French 34% · Mixed / Arabizi 13%
Listing intent: Seller 95.4% · Buyer 4.6%

All posts were collected from public Algerian Facebook real estate groups(using Facebook API in the Apify platforl) Phone numbers are replaced with [PHONE] before publishing (BIO tags preserved so the model learns positional context without memorizing digits)

---

Training Details

base_model:        algerianDeBERTa (DeBERTa-v2)
architecture:      DebertaV2ForTokenClassification
num_labels:        27  (BIO, 13 entity types)


max_seq_len:       192  
stride:            64    


optimizer:         AdamW
peak_lr:           2e-5 
llrd_factor:       0.9   
weight_decay:      0.01
adam_eps:          1e-6
adam_beta1:        0.9
adam_beta2:        0.999
max_grad_norm:     1.0
grad_accum_steps:  2     

epochs:            20 (early stop at epoch 14, patience=5)
warmup_ratio:      0.1
schedule:          cosine with warmup


label_smoothing:   0.05
class_weighting:   inverse-frequency, capped at 10× (rare tags: CITY, WILAYA, CONDITION)
dropout:           0.1  (attention + hidden)


best_val_f1:       0.9858  
test_f1:           0.9672
test_precision:    0.9566
test_recall:       0.9780

Training highlights

Layerwise Learning Rate Decay (LLRD): The classifier head trains at peak_lr=2e-5, each successive DeBERTa layer is scaled by 0.9×, reaching ≈4.3e-6 at the embedding layer, This preserves general language representations while aggressively adapting the top layers to the NER task

Weighted label-smoothed cross-entropy: Rare entity tags (CITY, WILAYA, CONDITION) carry up to 10× the loss weight of frequent tags , Label smoothing (ε=0.05) prevents the model from becoming over-confident on the abundant O tag

Sliding-window tokenization: Posts exceeding 192 tokens are split into overlapping chunks (stride=64). Predictions from overlapping windows are reconciled at entity boundaries, ensuring long posts are fully covered without truncation

---

Limitations

• NEIGHBORHOOD F1=0.85: Neighbourhood names in Algeria are highly variable in spelling across Arabic, French, and Arabizi. This entity is underrepresented in the training data (188 annotations). Performance will improve with more annotated data from underrepresented neighbourhoods
• PRICE edge cases: Non-standard price expressions that rely heavily on slang are occasionally missed, The model handles the most common formats reliably
• Platform distribution: Trained on Facebook posts — casual, informal register. May underperform on formal Arabic (MSA) or structured portal listings
• Purely extractive: This is a span classifier, not a generative model. It labels tokens; it does not summarise or rewrite listings

---

Intended Use

Use case	Notes
Structured extraction from classifieds	This is the Core use case , extract price, surface, location, type from raw posts
Real estate market analytics	Build price-per-m² indices by wilaya; track inventory trends
Lead enrichment pipelines	Enrich CRM records from social media listing text
Training data generation	Use model outputs as silver labels for downstream tasks
Algerian NLP research	Low-resource benchmark for Darja and Arabizi NER

---

Citation

If you use this model or the dataset in your research, please cite:

@misc{himeur2026algeriandeberta_ner,
  title        = {algerianDeBERTa-realestate-ner: Named Entity Recognition
                  for Algerian Real Estate Text in Darja, Arabizi, and French},
  author       = {Himeur, Ayoub},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/81melody/algerianDeBERTa-realestate-ner},
  note         = {Fine-tuned DeBERTa-v2 on annotated
                  Algerian Facebook real estate posts, 13 entity types}
}

---

License

Apache 2.0

Contact

[email protected]