Implementation of a Programming Language
Artin Mohammadi31 min read
I've been thinking about how cool it would be to implement a programming language from scratch for about a year! After a long time, I decided to leave all other projects at least for a while (even a startup company)...
Specifying the Desired Syntax
First of all, maybe I should talk about the name of the language! This is the timeline of the chosen names in order:
Ammepasand (in Farsi, it means something that everyone likes)
Amme (in Farsi, it means all people)
Farr (in Farsi, it means glory)
My initial idea was to create a language heavily inspired by Bash; a language with understandable and simple codes (I'm still not sure if Bash is a simple language?!) and can be used in the terminal for many tasks (such as manipulating strings and files, automating processes, etc.)...
And this was the chosen grammar for the Ammepsand language:
-- A worthless one-line comment
<code>
<code> -- A comment in front of a code
---
But this is a multi-line comment
for a longer explanation
---
<code>
-- Texts
var name = "Artin";
var about = m"Maybe a cool programming language
written with Python!"; -- With the `m` character before the double quote,
-- you can have a multi-line string...
-- Numbers
var age = 8;
var pi = 3.1415;
-- Data structures
var inspired_languages = list["Bash", "Scala", "Python"];
var the_developer = dict["name": "Artin", "age": 21];
println `5 + 1`; -- Output: 6
println `10 - 4`; -- Output: 6
println `2 * 3`; -- Output: 6
println `36 / 6`; -- Output: 6
println `6 ^ 1`; -- Output: 6
println `(10 * 60) / 100`; -- Output: 6
var x = 69;
var y = 85;
if x == y then -- Are they equal?
println "Have fun!";
elif x < y then -- Or `x` is less than `y`?
println "Congrats!";
else -- Anyway...
println "Sorry!";
end
-- Creating a simple loop with `for` and `seq`
for var i in seq 0..45 do -- From 0 to 43 itself
println "Suppress!";
end
-- Wait for `while`
while nil == nil do -- Repeats until `nil` is equal to `nil`
println "I love this language!";
end
fn foobar params[var blah1, var blah2] begins
if blah1 == blah2 then
println "You have two similar blahs!";
return true;
end
return false;
end
In the middle of the night, I said to myself, maybe I should make the syntax more similar to the C family... But after about 15 minutes, I saw that we were far away from Bash! After that, I decided to change the name of the language to Amme along with the syntax.
This was the calculator program I wrote in the fictional Amme language:
mut firstNumber;
mut secondNumber;
mut result;
while true = {
firstNumber = readLn("Enter the first number: ")
.toFloat();
secondNumber = readLn("Enter the second number: ")
.toFloat();
match readLn("Type the operator symbol: ") = {
for "+" = {
result = + firstNumber secondNumber;
} for "-" | "_" = {
result = - firstNumber secondNumber;
} for "*" | "x" | "×" = {
result = * firstNumber secondNumber;
} for "/" | "÷" = {
result = / firstNumber secondNumber;
} for "%" = {
result = % firstNumber secondNumber;
} for "^" | "**" = {
result = ^ firstNumber secondNumber;
} for ... = {
panic!("The operator is not supported!");
}
}
printLn("Result: #{result}");
}
It didn't take long for me to decide that we should have some rare features in the language we were going to implement:
Polish prefix notation for mathematical expressions
Starting indexes from one
Member-based structs
fn add(let x, let y) = {
return! + x y;
}
fn subtract(let x, let y) = {
return! - x y;
}
fn multiply(let x, let y) = {
return! * x y;
}
fn divide(let x, let y) = {
return! / x y;
}
fn modulus(let x, let y) = {
return! % x y;
}
fn power(let x, let y) = {
return! ^ x y;
}
let first_number = (
readln!("Enter the first number: ")
.tofloat()
);
let second_number = (
readln!("Enter the second number: ")
.tofloat()
);
for let op, let fn! in {
:"+" add, :"-" subtract, :"*" multiply,
:"/" divide, :"mod" modulus, :"^" power} = {
println(
"${first_number} ${op} ${second_number} = ${fn!(first_number, second_number)}"
);
}
Implementation
All parts cannot be explained in this post, I will only mention the challenges and solutions that came to my mind during that period of time.
Chopping the Code
For many people, the first solution that comes to mind for tokenization and even parsing is to use ANTLR, Bison, and other similar tools... I am not saying that it is a bad idea, but we are supposed to understand the process of creating a programming language from scratch; not just the implementation of several node visitors!
Instead of reading character by character and basic labeling, I went to use regular expressions... I really liked to implement the project as much as possible so that others can easily manipulate it; and this was the main reason for using regular expressions in this phase!
from dataclasses import dataclass, field
from typing import Optional, List
@dataclass(frozen=True)
class Token:
name: str
pattern: str
ignore: Optional[bool] = field(default=False, kw_only=True)
@dataclass(frozen=True)
class GroupedTokens:
pattern: str
tokens: List[Token]
@dataclass(frozen=True)
class TokenState:
row: int = field(kw_only=True)
column: int = field(kw_only=True)
name: str = field(kw_only=True)
value: str = field(kw_only=True)
A little further, you will understand what GroupedTokens is for!? And that I know that for a data class that all fields have a specific parameter, it can be defined in the decorator itself (e.g., @dataclass(frozen=True, kw_only=True))...
Our scanner logic:
import re
class RegexLexer:
"""A base class for building a lexer using regular expressions.
Attributes:
tokens: The language tokens.
_separator: A pattern that separates parts of the code.
_row: The current row position in the source code.
_column: And the column position.
"""
tokens: List[GroupedTokens]
def __init__(self) -> None:
# Patching the separator tokens pattern
self._separator = re.compile(
'|'.join(
f'({grouped_tokens.pattern})' for grouped_tokens in self.tokens
)
)
self._row = self._column = -1
def _match_token(self, chunk: str) -> Token:
"""Checks whether the token is valid or not."""
for grouped_tokens in self.tokens:
if re.fullmatch(grouped_tokens.pattern, chunk):
for token in grouped_tokens.tokens:
if re.fullmatch(token.pattern, chunk):
return token
raise ValueError(
'A strange thing was found! '
f'Line {self._row}, column {self._column}'
)
def _update_position(self, chunk: str) -> None:
"""Updates the pointer position after passing the current chunk."""
for char in chunk:
if char in '\r\n': # TODO: Check for different operating systems
self._row += 1
self._column = 1
else:
self._column += 1
return None
def tokenize(self, code: str) -> List[TokenState]:
"""Tokenizes the code and then labels them."""
self._row = self._column = 1
result = []
for chunk in filter(lambda x: x, self._separator.split(code)):
if not (token := self._match_token(chunk)).ignore:
result.append(
TokenState(
row=self._row,
column=self._column,
name=token.name,
value=chunk,
)
)
self._update_position(chunk)
return result
In the constructor, we create a pattern based on the groups defined by the developer(s) to break the code into smaller parts. The tokenize method is probably not that difficult to understand; but I must mention that it is much better to use generators instead of returning a list at the end of the function...
Here we define our own language tokens beautifully:
class FarrRegexLexer(RegexLexer):
tokens = [
GroupedTokens(
r'/{2}.*|\/\*[\s\S]*?\*\/',
[
Token('SingleLineComment', r'/{2}.*', ignore=True),
Token('MultiLineComment', r'\/\*[\s\S]*?\*\/', ignore=True),
],
),
GroupedTokens(
r'0[box]\d+|[\-\+]?(?:\d+\.(?!\.)\d*|\d*\.(?!\.)\d+|\d+)|'
r'r?"(?:[^"\\]|\\.)*"',
[
Token('Binary', r'0b\d+'),
Token('Octal', r'0o\d+'),
Token('Hexadecimal', r'0x\d+'),
Token('Integer', r'[\-\+]?\d+'),
Token('Float', r'[\-\+]?(?:\d+\.(?!\.)\d*|\d*\.(?!\.)\d+)'),
Token('String', r'r?"(?:[^"\\]|\\.)*"'),
],
),
GroupedTokens(
r'[\<\>]{2}\=|[&\|\=\:\+\-\<\>]{2}|[\<\>\!\+\-\*/%\^]\=|\.{2,3}|[\s\W]',
[
Token('LineBreaker', r'[\n\r]', ignore=True),
Token('Indent', r'[\040\t]', ignore=True),
Token('LeftParenthesis', r'\('),
Token('RightParenthesis', r'\)'),
Token('LeftBrace', r'\{'),
Token('RightBrace', r'\}'),
Token('LeftBracket', r'\['),
Token('RightBracket', r'\]'),
Token('Comma', r','),
Token('Dot', r'\.'),
Token('Colon', r'\:'),
Token('DoubleColon', r'\:{2}'),
Token('Increment', r'\+{2}'),
Token('Decrement', r'\-{2}'),
Token('Semicolon', r';'),
Token('Add', r'\+'),
Token('Subtract', r'\-'),
Token('Multiply', r'\*'),
Token('Divide', r'/'),
Token('Modulus', r'%'),
Token('Power', r'\^'),
Token('Not', r'\!'),
Token('And', r'&{2}'),
Token('Or', r'\|{2}'),
Token('Equal', r'\='),
Token('EqualEqual', r'\={2}'),
Token('NotEqual', r'\!\='),
Token('LeftShift', r'\<{2}'),
Token('RightShift', r'\>{2}'),
Token('LessThan', r'\<'),
Token('GreaterThan', r'\>'),
Token('LessThanOrEqual', r'\<\='),
Token('GreaterThanOrEqual', r'\>\='),
Token('LeftShiftEqual', r'\<{2}\='),
Token('RightShiftEqual', r'\>{2}\='),
Token('AddEqual', r'\+\='),
Token('SubtractEqual', r'\-\='),
Token('MultiplyEqual', r'\*\='),
Token('DivideEqual', r'/\='),
Token('ModulusEqual', r'%\='),
Token('PowerEqual', r'\^\='),
Token('Between', r'\.{2}'),
Token('Pass', r'\.{3}'),
],
),
GroupedTokens(
r'_?[A-Za-z][A-Za-z_]*\d{,3}(?:\?\!|\!\?|\!|\?)?|\w*',
[
Token('Null', r'null'),
Token('Use', r'use'),
Token('Variable', r'let'),
Token('If', r'if'),
Token('Else', r'else'),
Token('Match', r'match'),
Token('While', r'while'),
Token('Break', r'break!'),
Token('Continue', r'continue!'),
Token('For', r'for'),
Token('In', r'in'),
Token('Try', r'try'),
Token('Catch', r'catch'),
Token('Function', r'fn'),
Token('Return', r'return!'),
Token('Struct', r'struct'),
Token('Identifier', r'.*'),
],
),
]
Generating Syntax Trees
This part is very, very long, so I will only take small parts of it...
This is the base parser we use with a very simple logic:
from farr.lexer.base import TokenState
class Parser:
"""A brain that analyzes the grammar of a language.
Attributes:
_tokens_state: A list of TokenState objects.
_current_token: It is clear from the name.
_next_token: Read the previous attribute description.
"""
def __init__(self) -> None:
self._tokens_state = [] # type: ignore[var-annotated]
self._current_token = None
self._next_token = None
def at_end(self) -> bool:
"""Looks at the remaining tokens to check the end."""
return (
not self._tokens_state
and self._current_token is None
and self._next_token is None
)
def advance(self) -> Optional[TokenState]:
"""Advances to the next token in the tokens state."""
return self._tokens_state.pop(0) if self._tokens_state else None
def expect(self, *args: str) -> None:
"""Raises if the expectation is not met."""
if (args_ := '/'.join(args)) and self._current_token is None:
raise SyntaxError(f'Expected `{args_}`, but nothing here!')
elif self._current_token.name not in args: # type: ignore[attr-defined]
raise SyntaxError(
f'Expected `{args_}`, got `{self._current_token.name}`! ' # type: ignore[attr-defined]
f'Line {self._current_token.row}, column {self._current_token.column}'
)
return None
def check(self, *args: str) -> Optional[bool]:
"""Checks whether the current token matches the target or not."""
return (
self._current_token.name in args # type: ignore[attr-defined]
if self._current_token is not None
else None
)
def peek(self, *args: str) -> Optional[bool]:
"""Looks at the next token to match."""
return (
self._next_token.name in args # type: ignore[attr-defined]
if self._next_token is not None
else None
)
def step(self) -> None:
"""Moves the next and current token values forward."""
if (
self._tokens_state
and self._current_token is None
and self._next_token is None
):
self._next_token = self.advance() # type: ignore[assignment]
self._current_token = self._next_token
self._next_token = self.advance() # type: ignore[assignment]
def parse(self, tokens_state: List[TokenState]) -> ModuleNode:
"""Returns a AST that shows the structure of the code."""
raise NotImplementedError
We have over 65 nodes in Farr, but I'll only include 30% of them here to give you an idea of how they're structured and inherited:
from dataclasses import dataclass, field
from typing import Optional, Union, Any, List
class ASTNode:
pass
@dataclass
class BlockNode(ASTNode):
body: List[Union[ASTNode, Any]] = field(kw_only=True)
class ModuleNode(BlockNode):
pass
@dataclass
class PositionedNode(ASTNode):
row: int = field(kw_only=True)
column: int = field(kw_only=True)
class ExpressionNode(ASTNode):
pass
@dataclass
class HeterogeneousLiteralNode(PositionedNode, ExpressionNode):
value: str = field(kw_only=True)
class IntegerNode(HeterogeneousLiteralNode):
pass
class FloatNode(HeterogeneousLiteralNode):
pass
class StringNode(HeterogeneousLiteralNode):
pass
class IdentifierNode(HeterogeneousLiteralNode):
pass
@dataclass
class ItemizedExpressionNode(ExpressionNode):
items: List[
Union[
ExpressionNode,
'VariableDeclarationNode',
'AssignmentNode',
Any,
]
] = field(kw_only=True)
@dataclass
class CallNode(ExpressionNode):
invoke: IdentifierNode = field(kw_only=True)
args: ItemizedExpressionNode = field(kw_only=True)
class StatementNode(ASTNode):
pass
@dataclass
class VariableDeclarationNode(StatementNode):
identifier: IdentifierNode = field(kw_only=True)
expression: Optional[ExpressionNode] = field(kw_only=True)
class VariadicParameterDeclarationNode(VariableDeclarationNode):
pass
@dataclass
class AssignmentNode(StatementNode):
references: ItemizedExpressionNode = field(kw_only=True)
expression: ExpressionNode = field(kw_only=True)
@dataclass
class CodeUnitNode(StatementNode):
identifier: IdentifierNode = field(kw_only=True)
body: Optional[Union[BlockNode, ExpressionNode]] = field(kw_only=True)
@dataclass
class FunctionDefinitionNode(CodeUnitNode):
params: ItemizedExpressionNode = field(kw_only=True)
@dataclass
class MemberFunctionDefinitionNode(FunctionDefinitionNode):
struct: IdentifierNode = field(kw_only=True)
@dataclass
class StructDefinitionNode(CodeUnitNode):
parents: Optional[ItemizedExpressionNode] = field(kw_only=True)
A not so interesting solution that can be used to create the nodes needed to build a syntax tree is to reduce them to one; with fields such as node type, and its children... There are definitely different ideas for different phases; for example, another simple solution that can be used to provide nodes is to create general groups for them, such as literals, loops, etc.
Let's go to parse three literals (_parse_integer, _parse_string and _parse_identifier), a term (_parse_call) and a statement (_parse_function):
from functools import partial
class FarrParser(Parser):
def _parse_integer(self) -> IntegerNode:
"""Parses an integer token."""
self.expect('Integer')
integer = IntegerNode(
row=self._current_token.row, # type: ignore[attr-defined]
column=self._current_token.column, # type: ignore[attr-defined]
value=self._current_token.value, # type: ignore[attr-defined]
)
self.step()
return integer
def _parse_string(self) -> StringNode:
"""Parses a string token."""
self.expect('String')
string = StringNode(
row=self._current_token.row, # type: ignore[attr-defined]
column=self._current_token.column, # type: ignore[attr-defined]
value=self._current_token.value, # type: ignore[attr-defined]
)
self.step()
return string
def _parse_identifier(self) -> IdentifierNode:
"""Parses an identifier token."""
self.expect('Identifier', 'Symbol')
identifier = IdentifierNode(
row=self._current_token.row, # type: ignore[attr-defined]
column=self._current_token.column, # type: ignore[attr-defined]
value=normalize_identifier(
self._current_token.value # type: ignore[attr-defined]
),
)
self.step()
return identifier
def _resolve_call_argument(
self,
) -> Optional[Union[ExpressionNode, AssignmentNode]]:
"""Resolves a call argument expression."""
return (
self._parse_keyword_assignment()
if self.check('Identifier', 'Symbol') and self.peek('Equal')
else (
self._parse_expandable_argument()
if self.check('Pass')
and not self.peek('Comma', 'RightParenthesis')
else self._process_expression()
)
)
def _parse_call(self, invoke: IdentifierNode) -> CallNode:
"""Parses a function call expression."""
return CallNode(
invoke=invoke,
args=self._parenthesized( # type: ignore[arg-type]
partial(
self._comma_separated_items,
fn=self._resolve_call_argument,
)
),
)
def _resolve_parameter(
self,
) -> Optional[
Union[VariableDeclarationNode, VariadicParameterDeclarationNode]
]:
"""Resolves a parameter declaration."""
return (
self._parse_variadic_parameter_declaration()
if self.check('Variable')
and self._except_current_and_next_at(0, ('Pass',))
else (
self._parse_variable_declaration()
if self.check('Variable')
else None
)
)
def _parse_function(self) -> FunctionDefinitionNode:
"""Parses a function definition."""
self.expect('Function')
self.step()
return FunctionDefinitionNode(
identifier=self._parse_identifier(),
params=self._parenthesized( # type: ignore[arg-type]
partial(
self._comma_separated_items,
fn=self._resolve_parameter,
)
),
body=self._parse_block(self._process_expression_or_statement),
)
def parse(self, tokens_state: List[TokenState]) -> ModuleNode:
"""Parses the whole module and returns the root AST node."""
self._tokens_state = tokens_state
self.step()
return ModuleNode(
body=sum(
partition_a_sequence(
self._accumulate_until(
self.at_end,
self._process_expression_or_statement,
),
lambda x: isinstance(
x,
(
FunctionDefinitionNode,
StructDefinitionNode,
MemberFunctionDefinitionNode,
),
),
),
[],
)
)
As a programmer and maintainer of this language, I have to admit that Farr could have been smarter in parsing! I mean more in the part of combining terms (_process_expression)...
def _process_expression(self) -> Optional[ExpressionNode]:
"""Processes an expression."""
if (left := self._process_term()) is None:
return None
if self.check(
'EqualEqual',
'NotEqual',
'GreaterThan',
'LessThan',
'GreaterThanOrEqual',
'LessThanOrEqual',
):
operator = self._current_token
self.step()
left = RelationalOperationNode(
row=operator.row, # type: ignore[attr-defined]
column=operator.column, # type: ignore[attr-defined]
operator=operator.name, # type: ignore[attr-defined]
left=left,
right=self._validate( # type: ignore[arg-type]
self._process_expression,
('Expression',),
),
)
if self.check('And', 'Or'):
operator = self._current_token
self.step()
left = LogicalOperationNode(
row=operator.row, # type: ignore[attr-defined]
column=operator.column, # type: ignore[attr-defined]
operator=operator.name, # type: ignore[attr-defined]
left=left,
right=self._validate( # type: ignore[arg-type]
self._process_expression,
('Expression',),
),
)
if self.check('Increment'):
operator = self._current_token
self.step()
left = PostIncrementNode(
row=operator.row, # type: ignore[attr-defined]
column=operator.column, # type: ignore[attr-defined]
operator=None,
operand=(
left.expressions
if isinstance(left, ChainedExpressionsNode)
else ItemizedExpressionNode(items=[left])
),
)
elif self.check('Decrement'):
operator = self._current_token
self.step()
left = PostDecrementNode(
row=operator.row, # type: ignore[attr-defined]
column=operator.column, # type: ignore[attr-defined]
operator=None,
operand=(
left.expressions
if isinstance(left, ChainedExpressionsNode)
else ItemizedExpressionNode(items=[left])
),
)
if self.check('If'):
if_ = self._current_token
self.step()
condition = self._validate(
self._process_expression,
('Expression',),
)
self.expect('Else')
self.step()
left = TernaryOperationNode(
row=if_.row, # type: ignore[attr-defined]
column=if_.column, # type: ignore[attr-defined]
then=left,
condition=condition, # type: ignore[arg-type]
orelse=self._validate( # type: ignore[arg-type]
self._process_expression,
('Expression',),
),
)
return left
Code Execution
Oh, we have to go to the back-end part of our project! This part is both interesting and scary! Very interesting and very scary!
Preparing the environment for managing variables, functions, structs and methods; and the not so good base of the interpreter:
import sys
import re
from dataclasses import dataclass, field
from typing import Optional, Any, Dict
from farr.exceptions import (
BreakError,
ContinueError,
ReturnError,
InterpretError,
)
from farr.parser.nodes import ASTNode, ModuleNode
@dataclass
class Environment:
"""Manage scopes and their possible parents.
Attributes:
symbols: Current environment data independent of the parent.
parent: An optional parent.
"""
symbols: Optional[Dict[str, Any]] = field(
default_factory=dict, kw_only=True
)
parent: Optional['Environment'] = field(default=None, kw_only=True)
def assign(self, name: str, value: Any) -> None:
"""Assigns a value to a symbol in the current environment."""
self.symbols.update({name: value}) # type: ignore[union-attr]
def replace(self, name: str, value: Any) -> None:
"""Updates the symbol if it exists."""
if name in self.symbols: # type: ignore[operator]
self.symbols.update({name: value}) # type: ignore[union-attr]
return None
elif self.parent is not None:
return self.parent.replace(name, value)
raise NameError(f'Nothing was found with the name `{name}`!')
def locate(self, name: str) -> Any:
"""Tries to find the requested symbol."""
if (
value := self.symbols.get(name, None) # type: ignore[union-attr]
) is not None:
return value
elif self.parent is not None:
return self.parent.locate(name)
raise NameError(f'Nothing was found with the name `{name}`!')
def exists(self, name: str, depth: Optional[int] = 0) -> bool:
"""Checks if the symbol exists at the specified depth or not."""
environment = self
while depth > 0 and environment.parent is not None: # type: ignore[operator]
environment = environment.parent
depth -= 1 # type: ignore[operator]
return depth == 0 and name in environment.symbols # type: ignore[operator]
class Interpreter:
"""To walk on abstract syntax trees and execute their nodes.
Attributes:
builtin_symbols: A dictionary that includes the natives of the language.
environment: An instance of `Environment`.
"""
builtin_symbols: Dict[str, Any]
def __init__(self, *, environment: Optional[Environment] = None) -> None:
self.environment = (
environment
if environment is not None
else Environment(symbols=self.builtin_symbols.copy())
)
def _interpret(self, node: ASTNode) -> Any:
"""Interprets the given AST node."""
handler_name = '_interpret_{}'.format(
'_'.join(
map(
str.lower,
filter(
lambda x: x,
re.split(
r'([A-Z][a-z]*)',
node.__class__.__name__,
),
),
)
)
)
if (interpreter_method := getattr(self, handler_name, None)) is None:
raise AttributeError(
f'Implement the `{handler_name}` method in the '
f'`{self.__class__.__name__}` class...'
)
try:
return interpreter_method(node)
except (BreakError, ContinueError, ReturnError):
raise
except InterpretError as e:
raise InterpretError(
error=e.error, origin=e.origin # To avoid nesting errors
)
except BaseException as e:
raise InterpretError(error=e, origin=str(node))
def interpret(self, node: ModuleNode) -> None:
"""Tries to start the interpretation with caution."""
try:
self._interpret(node)
except InterpretError as e:
error_details = (
str(e.error).rstrip('!.') or 'No additional error details'
)
(first_row, first_column), *_ = re.findall(
r'row=(\d+), column=(\d+)', e.origin
)
print(
f'{e.error.__class__.__name__}: {error_details}! '
f'Around line {first_row}, column {first_column}.'
)
sys.exit(1)
This section can be very difficult to understand without getting into the code, especially for chained handle expressions (_process_chain_target) and calls (_populate_params)...
A half of the main code of the Farr language interpreter is as follows:
from functools import reduce
from farr.constants import (
RESOURCES_ROOT_PATH,
FILE_EXTENSION,
LIBRARY_INITIALIZER_FILE,
)
class FarrInterpreter(Interpreter):
builtin_symbols = {
'null': NullObject(),
'true': BooleanObject(value=True),
'false': BooleanObject(value=False),
'print': PythonNativePrintObject(),
'println': PythonNativePrintLineObject(),
'readln_e': PythonNativeReadLineObject(),
'panic_eq': PythonNativePanicObject(),
'assert_e': PythonNativeAssertObject(),
'exit_e': PythonNativeExitObject(),
'typeof_q': PythonNativeTypeOfObject(),
'cmd_eq': PythonNativeShellExecutionObject(),
'BaseError': PythonNativeBaseErrorObject,
'SystemExitError': PythonNativeSystemExitErrorObject,
'ArithmeticError': PythonNativeArithmeticErrorObject,
'AttributeError': PythonNativeAttributeErrorObject,
'ImportError': PythonNativeImportErrorObject,
'LookupError': PythonNativeLookupErrorObject,
'NameError': PythonNativeNameErrorObject,
'OSError': PythonNativeOSErrorObject,
'NotImplementedError': PythonNativeNotImplementedErrorObject,
'TypeError': PythonNativeTypeErrorObject,
'ValueError': PythonNativeValueErrorObject,
}
def _interpret_module_node(self, node: ModuleNode) -> None:
"""Interprets a `ModuleNode`."""
for child in node.body:
self._interpret(child)
return None
def _interpret_null_node(self, node: NullNode) -> NullObject:
"""Returns a `NullObject`."""
return NullObject()
def _interpret_integer_node(self, node: IntegerNode) -> IntegerObject:
"""Converts an `IntegerNode` to an `IntegerObject`."""
return IntegerObject(value=int(node.value))
def _interpolate(self, match_: re.Match) -> str:
"""Interpolates and executes a match."""
return ' '.join(
map(
lambda x: str(self._interpret(x)),
FarrParser()
.parse(FarrRegexLexer().tokenize(f'{match_.group(1)};'))
.body,
)
)
def _interpret_string_node(self, node: StringNode) -> StringObject:
"""Converts a `StringNode` to a `StringObject`."""
return StringObject(
value=re.sub(
r'(?<!\\)\$\{(.*?)\}',
self._interpolate,
(
cleaned_value.translate(
str.maketrans(
{ # type: ignore[arg-type]
'\n': r'\\n',
'\t': r'\\t',
'\b': r'\\b',
'\r': r'\\r',
'\"': r'\\\"',
'\\': r'\\\\',
}
)
)
if (
(cleaned_value := re.sub(r'^r?"|"$', '', node.value))
and node.value.startswith('r')
)
else re.sub(
r'\\([ntrb"\\])',
lambda match_: {
'n': '\n',
't': '\t',
'r': '\r',
'b': '\b',
'"': '"',
'\\': '\\',
}.get(
match_.group(1), None # type: ignore[arg-type]
),
cleaned_value,
)
),
)
)
def _process_chain_target(
self,
x: Union[ASTNode, FarrObject],
y: ASTNode,
) -> FarrObject:
"""Handles the next part of the chain."""
if (
result := self._interpret(x) if isinstance(x, ASTNode) else x
) and isinstance(y, IdentifierNode):
if isinstance(
target := getattr(result, y.value), NonPythonNativeObject
):
target.environment = result.environment
return (
PythonNativeClassMethodObject(method=target)
if isinstance(target, types.MethodType)
else target
)
elif isinstance(y, CallNode):
if isinstance(
target := getattr(result, y.invoke.value), NonPythonNativeObject
):
target.environment = result.environment
return (
self._call_python_native_object(target, y.args) # type: ignore[return-value]
if isinstance(target, types.MethodType)
else self._call_non_python_native_object(target, y.args)
)
return result[self._interpret(y)]
def _interpret_chained_expressions_node(
self,
node: ChainedExpressionsNode,
) -> FarrObject:
"""Returns the result of a chain of expressions."""
return reduce(self._process_chain_target, node.expressions.items) # type: ignore[return-value, arg-type]
def _populate_params(
self,
params: ItemizedExpressionNode,
args: ItemizedExpressionNode,
) -> None:
"""Tries to assign arguments to parameters."""
required, optional = partition_a_sequence(
params.items, lambda x: x.expression is None
)
required, variadic = partition_a_sequence(
required,
lambda x: not isinstance(x, VariadicParameterDeclarationNode),
)
args_, kwargs = partition_a_sequence(
args.items, lambda x: not isinstance(x, AssignmentNode)
)
args_, exp_args = partition_a_sequence(
args_, lambda x: not isinstance(x, ExpandableArgumentNode)
)
if len(args.items) > len(params.items) and not variadic:
raise TypeError(
'Too many arguments provided. '
'Please check the function definition.'
)
elif len(args.items) < len(required) and not variadic and not exp_args:
raise TypeError(
'Not enough arguments provided for the required parameters.'
)
elif len(args_) > len(required) and not variadic:
raise TypeError(
'Provided more positional arguments than the function accepts.'
)
elif not args_ and not exp_args and required:
raise TypeError('Required parameters are missing arguments.')
elif (args_ or exp_args) and not required and optional:
raise TypeError(
'Positional arguments provided but the function expects '
'only keyword arguments.'
)
elif kwargs and not optional:
raise TypeError(
'Keyword arguments provided but the function does not accept them.'
)
elif kwargs and set(
map(lambda x: x.references.items[0].value, kwargs)
).isdisjoint(map(lambda x: x.identifier.value, optional)):
raise TypeError(
'Provided keyword arguments do not correspond to any optional '
'parameters. Please check the function definition for valid '
'parameter names.'
)
elif len(variadic) > 1:
raise TypeError(
'Multiple variadic parameters defined. '
'A function can only have one variadic parameter.'
)
for param, arg in zip(required.copy(), args_.copy()):
required.pop(0)
args_.pop(0)
self.environment.assign(param.identifier.value, arg)
if (
exp_args
and (exp_args := exp_args.pop(0).expression)
and variadic
and not required
):
self.environment.assign(variadic.pop(0).identifier.value, exp_args)
elif (
exp_args
and len(exp_args.elements) == len(required) # type: ignore[attr-defined]
and not variadic
):
for param, arg in zip(required, exp_args):
self.environment.assign(param.identifier.value, arg)
elif exp_args and required:
for param, arg in zip(required.copy(), exp_args.elements.copy()): # type: ignore[attr-defined]
required.pop(0)
exp_args.elements.pop(0) # type: ignore[attr-defined]
self.environment.assign(param.identifier.value, arg)
if exp_args and not variadic:
raise TypeError(
'Extra arguments remain after unpacking, but no variadic '
'parameter is available to receive them.'
)
variadic_ = self.environment.locate(
variadic.pop(0).identifier.value
)
variadic_.elements.extend(exp_args)
elif args_ and variadic:
variadic_ = self.environment.locate(
variadic.pop(0).identifier.value
)
variadic_.elements.extend(args_)
for kwarg in kwargs:
if not self.environment.exists(
name := kwarg.references.items.copy().pop().value, 0
):
raise NameError(f'There is no parameter name `{name}`!')
self.environment.assign(name, kwarg.expression)
return None
def _call_non_python_native_object(
self,
invoke: NonPythonNativeObject,
args: ItemizedExpressionNode,
) -> FarrObject:
"""Calls native objects of our language."""
args_, kwargs = partition_a_sequence(
args.items, lambda x: not isinstance(x, AssignmentNode)
)
args_, exp_args = partition_a_sequence(
args_, lambda x: not isinstance(x, ExpandableArgumentNode)
)
args_ = list(map(self._interpret, args_))
exp_args = list(
map(
lambda x: ExpandableArgumentNode(
expression=self._interpret(x.expression)
),
exp_args,
)
)
kwargs = list(
map(
lambda x: AssignmentNode(
references=x.references,
expression=self._interpret(x.expression),
),
kwargs,
)
)
environment_backup = self.environment
self.environment = Environment(
parent=(
invoke.environment
if invoke.environment is not None
else self.environment
)
)
self._interpret(
invoke.params
if isinstance(invoke, FunctionDefinitionObject)
else invoke.attributes # type: ignore[attr-defined]
)
self._populate_params(
(
invoke.params
if isinstance(
invoke,
FunctionDefinitionObject,
)
else invoke.attributes # type: ignore[attr-defined]
),
ItemizedExpressionNode(items=sum([args_, exp_args, kwargs], [])),
)
try:
self._interpret(invoke.body)
except ReturnError as e:
result = e.expression
else:
result = (
StructInstanceObject(environment=self.environment.copy())
if isinstance(invoke, StructDefinitionObject)
else NullObject()
)
self.environment = environment_backup
return result # type: ignore[return-value]
def _call_python_native_object(
self,
invoke: PythonNativeObject,
args: ItemizedExpressionNode,
) -> Optional[FarrObject]:
"""Calls Python native objects with respect to arguments."""
args_, kwargs = partition_a_sequence(
args.items, lambda x: not isinstance(x, AssignmentNode)
)
return invoke( # type: ignore[operator]
*map(self._interpret, args_),
**dict(
map(
lambda x: (
x.references.items.copy().pop(0).value,
self._interpret(x.expression),
),
kwargs,
)
),
)
def _interpret_call_node(self, node: CallNode) -> FarrObject:
"""Calls a callable object with the taken arguments."""
return (
self._call_non_python_native_object(invoke, node.args) # type: ignore[return-value]
if isinstance(
invoke := self._interpret(node.invoke), NonPythonNativeObject
)
else self._call_python_native_object(invoke, node.args)
)
def _interpret_pre_increment_node(
self,
node: PreIncrementNode,
) -> Union[IntegerObject, FloatObject]:
"""Adds one unit to the previous value and returns it."""
*pointers, target = node.operand.items # type: ignore[attr-defined]
if not pointers:
self.environment.replace(
target.value,
result := self.environment.locate(target.value)
+ IntegerObject(value=1),
)
return result
pointer = self._interpret(pointers.pop(0))
while pointers:
pointer = (
getattr(pointer, link.value) # type: ignore[union-attr]
if not isinstance(link := pointers.pop(0), RangeNode)
else pointer[self._interpret(link)]
)
if not hasattr(pointer, 'environment'):
pointer[target] = (
result := (
pointer[target := self._interpret(target)]
+ IntegerObject(value=1)
)
)
return result
pointer.environment.replace(
target.value,
result := pointer.environment.locate(target.value)
+ IntegerObject(value=1),
)
return result
def _interpret_post_increment_node(
self,
node: PostIncrementNode,
) -> Union[IntegerObject, FloatObject]:
"""Returns the previous value and then adds one to it."""
*pointers, target = node.operand.items # type: ignore[attr-defined]
if not pointers:
self.environment.replace(
target.value,
(result := self.environment.locate(target.value))
+ IntegerObject(value=1),
)
return result
pointer = self._interpret(pointers.pop(0))
while pointers:
pointer = (
getattr(pointer, link.value) # type: ignore[union-attr]
if not isinstance(link := pointers.pop(0), RangeNode)
else pointer[self._interpret(link)]
)
if not hasattr(pointer, 'environment'):
pointer[target] = (
result := pointer[target := self._interpret(target)]
) + IntegerObject(value=1)
return result
pointer.environment.replace(
target.value,
(result := pointer.environment.locate(target.value))
+ IntegerObject(value=1),
)
return result
def _interpret_arithmetic_operation_node(
self,
node: ArithmeticOperationNode,
) -> Optional[HeterogeneousLiteralObject]:
"""Interprets a mathematical operation."""
left = self._interpret(node.left)
right = self._interpret(node.right)
match node.operator:
case 'LeftShift':
return left << right
case 'RightShift':
return left >> right
case 'Add':
return left + right
case 'Subtract':
return left - right
case 'Multiply':
return left * right
case 'Divide':
return left / right
case 'Modulus':
return left % right
case 'Power':
return left**right
return None
def _resolve_import_path(
self,
path: ItemizedExpressionNode,
) -> Union[pathlib.Path, List[pathlib.Path]]:
"""Validates and returns the required path(s)."""
if (
resources_root_path := os.getenv(RESOURCES_ROOT_PATH, None)
) is None:
raise OSError(
f'The `{RESOURCES_ROOT_PATH}` environment variable is not set.'
)
target = pathlib.Path(f'{resources_root_path}/libs')
for part in map(lambda x: x.value, path.items): # type: ignore[union-attr]
target /= part
if (
not target.is_dir()
and not target.with_suffix(f'.{FILE_EXTENSION}').is_file()
):
raise OSError(
f'There is no file or folder named `{target}(.{FILE_EXTENSION})`!'
)
elif (
target.is_dir()
and not (target / LIBRARY_INITIALIZER_FILE).is_file()
):
raise ImportError(
f'There is no `{LIBRARY_INITIALIZER_FILE}` in `{target}` directory...'
)
return (
list(filter(lambda x: x.is_file(), target.iterdir()))
if target.is_dir()
else target.with_suffix(f'.{FILE_EXTENSION}')
)
def _interpret_use_node(self, node: UseNode) -> None:
"""Handles the import of a library or module."""
if issubclass(
(resolved_path := self._resolve_import_path(node.path)).__class__,
pathlib.Path,
):
self.environment.assign(
resolved_path.stem,
ModuleObject(
environment=self._create_module_environment(resolved_path)
),
)
return None
(library_initializer, *_), modules_path = partition_a_sequence(
resolved_path, lambda x: x.name == LIBRARY_INITIALIZER_FILE
)
library_environment = self._create_module_environment(
library_initializer
)
for module_path in modules_path:
library_environment.assign(
module_path.stem,
ModuleObject(
environment=self._create_module_environment(module_path)
),
)
self.environment.assign(
library_initializer.parent.stem,
LibraryObject(environment=library_environment),
)
def _interpret_variable_declaration_node(
self,
node: VariableDeclarationNode,
) -> None:
"""Defines a variable in the environment."""
self.environment.assign(
node.identifier.value,
(
self._interpret(node.expression)
if node.expression is not None
else NullObject()
),
)
def _interpret_variadic_parameter_declaration_node(
self,
node: VariadicParameterDeclarationNode,
) -> None:
"""Interprets a variadic parameter declaration node."""
self.environment.assign(
node.identifier.value,
(
self._interpret(node.expression)
if node.expression is not None
else ListObject(elements=[])
),
)
def _interpret_assignment_node(self, node: AssignmentNode) -> None:
"""Updates the content of a variable."""
*pointers, target = node.references.items
if not pointers:
self.environment.replace(
target.value, self._interpret(node.expression) # type: ignore[union-attr]
)
return None
pointer = self._interpret(pointers.pop(0))
while pointers:
pointer = (
getattr(pointer, link.value) # type: ignore[union-attr]
if not isinstance(link := pointers.pop(0), RangeNode)
else pointer[self._interpret(link)]
)
if not hasattr(pointer, 'environment'):
pointer[self._interpret(target)] = self._interpret(node.expression)
return None
pointer.environment.replace(
target.value, self._interpret(node.expression) # type: ignore[union-attr]
)
def _interpret_while_node(self, node: WhileNode) -> None:
"""Interprets a `WhileNode`."""
while self._interpret(node.condition):
try:
self._interpret(node.body)
except BreakError:
break
except ContinueError:
continue
else:
if node.orelse is not None:
self._interpret(node.orelse)
return None
def _interpret_if_node(self, node: IfNode) -> None:
"""Interprets an `IfNode`."""
if self._interpret(node.condition):
self._interpret(node.body)
elif node.orelse is not None:
self._interpret(node.orelse)
return None
def _interpret_function_definition_node(
self,
node: FunctionDefinitionNode,
) -> None:
"""Defines a `FunctionDefinitionObject` in the environment."""
self.environment.assign(
node.identifier.value,
FunctionDefinitionObject(
body=node.body, # type: ignore[arg-type]
params=node.params,
),
)
def _interpret_member_function_definition_node(
self,
node: MemberFunctionDefinitionNode,
) -> None:
"""Adds a `FunctionDefinitionNode` to the body of the struct."""
struct = self.environment.locate(node.struct.value)
struct.body.body.append(
FunctionDefinitionNode(
identifier=node.identifier,
body=node.body,
params=node.params,
)
)
def _populate_on_parents(
self,
body: BlockNode,
parents: Optional[ItemizedExpressionNode],
) -> Tuple[BlockNode, ItemizedExpressionNode]:
"""Collects all parent properties."""
attributes = (
body.body.pop() if body.body else ItemizedExpressionNode(items=[])
)
for parent in map(
lambda x: self.environment.locate(x.value), # type: ignore[union-attr]
parents.items if parents is not None else [],
):
attributes.items = parent.attributes.items + attributes.items # type: ignore[union-attr]
body.body = parent.body.body + body.body
return body, attributes # type: ignore[return-value]
def _interpret_struct_definition_node(
self,
node: StructDefinitionNode,
) -> None:
"""Defines a `StructDefinitionObject` in the environment."""
body, attributes = self._populate_on_parents(node.body, node.parents) # type: ignore[arg-type]
self.environment.assign(
node.identifier.value,
StructDefinitionObject(
body=body,
attributes=attributes,
),
)
I can say that my biggest problem in implementing the interpreter was my inability to understand the logic of matching arguments with parameters! I was involved in the implementation of the interpreter for about a week...
Some objects that the language provides as its native:
class FarrObject:
pass
class ExpressionObject(FarrObject):
pass
class NullObject(ExpressionObject):
def __str__(self) -> str:
"""Returns `null`."""
return 'null'
def __bool__(self) -> bool:
"""Returns the essence of `NullObject`."""
return False
def __hash__(self) -> int:
"""Returns zero as the object hash."""
return 0
def __eq__(self, other: FarrObject) -> 'BooleanObject': # type: ignore[override]
"""Compares the equality of nothing with another object."""
return BooleanObject(value=False == other) # noqa: E712
def __ne__(self, other: FarrObject) -> 'BooleanObject': # type: ignore[override]
"""Compares the inequality of nothing with another object."""
return BooleanObject(value=False != other) # noqa: E712
@dataclass
class HeterogeneousLiteralObject(ExpressionObject):
value: Any = field(kw_only=True)
def __str__(self) -> str:
"""Returns the value as a `str`."""
return str(self.value)
def __bool__(self) -> bool:
"""Returns the value status as `bool`."""
return bool(self.value)
def __hash__(self) -> int:
"""Calculates the hash of the object."""
return hash(self.value)
def __eq__(self, other: FarrObject) -> 'BooleanObject': # type: ignore[override]
"""Checks whether the two values are equal or not."""
return BooleanObject(value=self.value == other)
def __ne__(self, other: FarrObject) -> 'BooleanObject': # type: ignore[override]
"""Checks if the two values are not equal."""
return BooleanObject(value=self.value != other)
def __lt__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> 'BooleanObject':
"""Checks if it is a smaller value or not."""
if not isinstance(self, (IntegerObject, FloatObject)) or not isinstance(
other, (IntegerObject, FloatObject)
):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `<` with type `{other.__class__.__name__}`!'
)
return BooleanObject(value=self.value < other.value)
def __gt__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> 'BooleanObject':
"""Checks if the value is greater than or not."""
if not isinstance(self, (IntegerObject, FloatObject)) or not isinstance(
other, (IntegerObject, FloatObject)
):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `>` with type `{other.__class__.__name__}`!'
)
return BooleanObject(value=self.value > other.value)
def __le__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> 'BooleanObject':
"""Checks if the value is less than or equal to or not."""
if not isinstance(self, (IntegerObject, FloatObject)) or not isinstance(
other, (IntegerObject, FloatObject)
):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `<=` with type `{other.__class__.__name__}`!'
)
return BooleanObject(value=self.value <= other.value)
def __ge__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> 'BooleanObject':
"""Checks if the value is greater than or equal to or not."""
if not isinstance(self, (IntegerObject, FloatObject)) or not isinstance(
other, (IntegerObject, FloatObject)
):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `>=` with type `{other.__class__.__name__}`!'
)
return BooleanObject(value=self.value >= other.value)
def isin(self, list_: 'ListObject') -> 'BooleanObject':
"""Checks the existence of the object value in the list."""
return BooleanObject(value=self.value in list_)
class IntegerObject(HeterogeneousLiteralObject):
def __lshift__(self, other: 'IntegerObject') -> 'IntegerObject':
"""Performs bitwise left shift."""
if not isinstance(other, IntegerObject):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `<<` with type `{other.__class__.__name__}`!'
)
return IntegerObject(value=self.value << other.value)
def __rshift__(self, other: 'IntegerObject') -> 'IntegerObject':
"""Performs bitwise right shift."""
if not isinstance(other, IntegerObject):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `>>` with type `{other.__class__.__name__}`!'
)
return IntegerObject(value=self.value >> other.value)
def __add__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> Union['IntegerObject', 'FloatObject']:
"""Adds the two values together."""
if not isinstance(other, (IntegerObject, FloatObject)):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `+` with type `{other.__class__.__name__}`!'
)
return other.__class__(value=self.value + other.value)
def __sub__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> Union['IntegerObject', 'FloatObject']:
"""Subtracts two existing values."""
if not isinstance(other, (IntegerObject, FloatObject)):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `-` with type `{other.__class__.__name__}`!'
)
return other.__class__(value=self.value - other.value)
def __mul__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> Union['IntegerObject', 'FloatObject']:
"""Multiplies two existing values together."""
if not isinstance(other, (IntegerObject, FloatObject)):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `*` with type `{other.__class__.__name__}`!'
)
return other.__class__(value=self.value * other.value)
def __truediv__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> 'FloatObject':
"""Divides the existing values."""
if not isinstance(other, (IntegerObject, FloatObject)):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `/` with type `{other.__class__.__name__}`!'
)
return FloatObject(value=self.value / other.value)
def __mod__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> Union['IntegerObject', 'FloatObject']:
"""Calculates the remainder of the division."""
if not isinstance(other, (IntegerObject, FloatObject)):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `%` with type `{other.__class__.__name__}`!'
)
return other.__class__(value=self.value % other.value)
def __pow__(
self,
other: Union['IntegerObject', 'FloatObject'],
) -> Union['IntegerObject', 'FloatObject']:
"""Calculates the exponentiation."""
if not isinstance(other, (IntegerObject, FloatObject)):
raise TypeError(
f'Type `{self.__class__.__name__}` does not support '
f'operator `^` with type `{other.__class__.__name__}`!'
)
return other.__class__(value=self.value**other.value)
def tostring(self) -> 'StringObject':
"""Converts the value of the object to a string."""
return StringObject(value=str(self))
class StringObject(HeterogeneousLiteralObject):
def __getitem__(self, key: 'RangeObject') -> 'StringObject':
"""Returns the characters in the string based on range."""
if key.from_.value <= 0 or key.by is not None and key.by.value <= 0: # type: ignore[union-attr]
raise IndexError('Non-positive indexes are not allowed!')
return (
StringObject(value=self.value[key.from_.value - 1]) # type: ignore[union-attr]
if key.to is None and key.by is None
else StringObject(
value=self.value[
key.from_.value # type: ignore[union-attr]
- 1 : key.to.value if key.to is not None else None : (
key.by.value if key.by is not None else None
)
]
)
)
def __iter__(self) -> 'StringObject':
"""Iterates over the characters of the string."""
self._index = 0
return self
def __next__(self) -> 'StringObject':
"""Returns the next character."""
if self._index >= len(self.value):
raise StopIteration
char = self.value[self._index]
self._index += 1
return char
def toint(self) -> IntegerObject:
"""Converts the value to an integer."""
return IntegerObject(value=int(self.value))
def tofloat(self) -> FloatObject:
"""Converts the value to a decimal number."""
return FloatObject(value=float(self.value))
def tolower(self) -> 'StringObject':
"""Converts the value to lowercase letters."""
return StringObject(value=self.value.lower())
def toupper(self) -> 'StringObject':
"""Converts the value to uppercase."""
return StringObject(value=self.value.upper())
def concat(self, object_: FarrObject) -> 'StringObject':
"""Merges the string with another object."""
return StringObject(value=self.value + str(object_))
def split(self, separator: Optional['StringObject'] = None) -> 'ListObject':
"""Separates the string based on the separator."""
return ListObject(
elements=list(
map(
lambda x: StringObject(value=x),
(
filter(lambda x: x, self.value.split(separator.value))
if separator is not None
else self.value
),
)
)
)
def removeprefix(self, prefix: 'StringObject') -> 'StringObject':
"""Removes a prefix from the string if it exists."""
return StringObject(value=self.value.removeprefix(prefix.value))
def removesuffix(self, suffix: 'StringObject') -> 'StringObject':
"""Removes a suffix from the string if it exists."""
return StringObject(value=self.value.removesuffix(suffix.value))
def count_q(self, subset: 'StringObject') -> IntegerObject:
"""Returns the number of matches by subset."""
return IntegerObject(value=self.value.count(subset.value))
def nearest_q(self, subset: 'StringObject') -> IntegerObject:
"""Returns the index of the first matched item."""
return IntegerObject(
value=(
result + 1
if (result := self.value.find(subset.value)) != -1
else -1
)
)
def contains_q(self, subset: 'StringObject') -> BooleanObject:
"""Returns whether the given subset exists in the string."""
return BooleanObject(value=subset.value in self.value)
def startswith_q(self, prefix: 'StringObject') -> BooleanObject:
"""Returns true if the beginning of the string is the same as the input value."""
return BooleanObject(value=self.value.startswith(prefix.value))
def endswith_q(self, suffix: 'StringObject') -> BooleanObject:
"""Returns true if the end of the string is the same as the input value."""
return BooleanObject(value=self.value.endswith(suffix.value))
@dataclass
class RangeObject(ExpressionObject):
from_: Optional[IntegerObject] = field(kw_only=True)
to: Optional[IntegerObject] = field(default=None, kw_only=True)
by: Optional[IntegerObject] = field(default=None, kw_only=True)
def __str__(self) -> str:
"""Returns the object as a string."""
return (
f'[{self.from_}, {self.by if self.by is not None else 1}'
f'..{self.to if self.to is not None else "undefined"}]'
)
def __hash__(self) -> int:
"""Calculates the hash of the object."""
return hash((self.from_, self.to, self.by))
def __iter__(self) -> 'RangeObject':
"""Iterates over the range defined by the object."""
self._number = self.from_
return self
def __next__(self) -> int:
"""Returns the next integer in the range."""
if self.to is not None and self._number > self.to: # type: ignore[operator]
raise StopIteration
result = self._number
self._number += ( # type: ignore[operator, assignment]
self.by if self.by is not None else IntegerObject(value=1)
)
return result # type: ignore[return-value]
class DataStructureObject(ExpressionObject):
pass
@dataclass
class ListObject(DataStructureObject):
elements: List[Optional[FarrObject]] = field(kw_only=True)
def __str__(self) -> str:
"""Returns elements separated by a semicolon."""
return '; '.join(map(str, self.elements))
def __hash__(self) -> int:
"""Returns the object ID as a hash."""
return id(self)
def __getitem__(
self,
key: 'RangeObject',
) -> FarrObject:
"""Extracts a range of elements."""
if key.from_.value <= 0 or key.by is not None and key.by.value <= 0: # type: ignore[union-attr]
raise IndexError('Non-positive indexes are not allowed!')
return (
self.elements[key.from_.value - 1] # type: ignore[return-value, union-attr]
if key.to is None and key.by is None
else ListObject(
elements=self.elements[
key.from_.value # type: ignore[union-attr]
- 1 : key.to.value if key.to is not None else None : (
key.by.value if key.by is not None else None
)
]
)
)
def __setitem__(
self,
key: 'RangeObject',
value: FarrObject,
) -> None:
"""Updates the elements based on the given range."""
if key.from_.value <= 0 or key.by is not None and key.by.value <= 0: # type: ignore[union-attr]
raise IndexError('Non-positive indexes are not allowed!')
elif key.to is None and key.by is None:
self.elements[key.from_.value - 1] = value # type: ignore[union-attr]
return None
self.elements[ # type: ignore[call-overload]
key.from_.value # type: ignore[union-attr]
- 1 : key.to.value if key.to is not None else None : (
key.by.value if key.by is not None else None
)
] = value
def __iter__(self) -> 'ListObject':
"""Iterates the elements in the list."""
self._index = 0
return self
def __next__(self) -> FarrObject:
"""Returns the next element of the list."""
if self._index >= len(self.elements):
raise StopIteration
element = self.elements[self._index]
self._index += 1
return element # type: ignore[return-value]
@property
def first(self) -> FarrObject:
"""Returns the first element if the list is not empty."""
if not self.elements:
raise IndexError('The list is empty!')
return self.elements[0] # type: ignore[return-value]
@property
def last(self) -> FarrObject:
"""Returns the last element if the list is not empty."""
if not self.elements:
raise IndexError('The list is empty!')
return self.elements[-1] # type: ignore[return-value]
@property
def length(self) -> IntegerObject:
"""Returns the number of elements in the list."""
return IntegerObject(value=len(self.elements))
def isempty_q(self) -> BooleanObject:
"""Returns the status of the list being empty or not."""
return BooleanObject(value=not bool(self.elements))
def clear_e(self) -> NullObject:
"""Removes all elements from the list."""
self.elements = []
return NullObject()
def nearest_q(self, element: FarrObject) -> IntegerObject:
"""Returns the index of the closest element found in the list."""
return IntegerObject(
value=(
self.elements.index(element) + 1
if element in self.elements
else -1
)
)
def iprepend_e(self, element: FarrObject) -> NullObject:
"""Adds an element to the beginning of the list."""
self.elements.insert(0, element)
return NullObject()
def iappend_e(self, element: FarrObject) -> NullObject:
"""Adds an element to the end of the list"""
self.elements.append(element)
return NullObject()
def pop_e(self, index: IntegerObject) -> FarrObject:
"""Deletes an element based on the index."""
if index.value <= 0:
raise IndexError(
'Using an index smaller than or equal to zero is not allowed!'
)
return self.elements.pop(index.value - 1) # type: ignore[return-value]
def popitem_e(self, value: FarrObject) -> FarrObject:
"""Discards an element based on the given value."""
return self.elements.pop(self.elements.index(value)) # type: ignore[return-value]
def reverse(self) -> 'ListObject':
"""Returns the reversed list."""
return ListObject(elements=list(reversed(self.elements))) # type: ignore[type-var]
def ireverse_e(self) -> 'ListObject':
"""Reverses the list and returns the new state."""
self.elements = list(reversed(self.elements)) # type: ignore[type-var]
return self
def sort(self) -> 'ListObject':
"""Returns the sorted list."""
return ListObject(elements=sorted(self.elements)) # type: ignore[type-var]
def isort_e(self) -> 'ListObject':
"""Sorts the list in its own place."""
self.elements = sorted(self.elements) # type: ignore[type-var]
return self
def shuffle(self) -> 'ListObject':
"""Returns a shuffled list."""
return ListObject(
elements=sorted(self.elements, key=lambda _: random.random())
)
def ishuffle_e(self) -> 'ListObject':
"""Assigns the shuffled list to the object and then returns it."""
self.elements = sorted(self.elements, key=lambda _: random.random())
return self
def join(self, separator: Optional[StringObject] = None) -> StringObject:
"""Merges elements together."""
return StringObject(
value=(separator.value if separator is not None else '').join(
map(str, self.elements)
)
)
class PythonNativeObject(ExpressionObject):
pass
@dataclass
class PythonNativeClassMethodObject(PythonNativeObject):
method: types.MethodType = field(kw_only=True)
def __eq__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks the sameness of two methods."""
return BooleanObject(value=self.method.__func__.__qualname__ == other)
def __ne__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks to see the difference between the two methods."""
return BooleanObject(value=self.method.__func__.__qualname__ != other)
def __call__(
self,
*args: Tuple[FarrObject, ...],
**kwargs: Dict[str, FarrObject],
) -> FarrObject:
"""Calls the method."""
return self.method(*args, **kwargs)
class PythonNativePrintObject(PythonNativeObject):
def __call__(
self,
*args: Tuple[FarrObject, ...],
) -> NullObject:
"""Prints and stays on the same line."""
print(*args, end='')
return NullObject()
class PythonNativePrintLineObject(PythonNativeObject):
def __call__(
self,
*args: Tuple[FarrObject, ...],
) -> NullObject:
"""Prints and goes to the next line."""
print(*args)
return NullObject()
class PythonNativeReadLineObject(PythonNativeObject):
def __call__(self, prompt: Optional[StringObject] = None) -> StringObject:
"""Takes an input from the user."""
return StringObject(value=input(prompt if prompt is not None else ''))
class PythonNativePanicObject(PythonNativeObject):
def __call__(
self,
exception: Optional[BaseException] = None,
) -> None:
"""Throws an error."""
raise exception if exception is not None else BaseException # type: ignore[misc]
class PythonNativeAssertObject(PythonNativeObject):
def __call__(
self,
condition: FarrObject,
message: Optional[StringObject] = None,
) -> None:
"""Panics if the condition is not correct."""
assert condition, message if message is not None else ''
class PythonNativeExitObject(PythonNativeObject):
def __call__(self, code: Optional[IntegerObject] = None) -> None:
"""Comes out based on the given exit code."""
sys.exit(code) # type: ignore[arg-type]
class PythonNativeTypeOfObject(PythonNativeObject):
def __call__(self, object_: FarrObject) -> StringObject:
"""Returns the object type."""
return StringObject(value=object_.__class__.__name__)
class PythonNativeShellExecutionObject(PythonNativeObject):
def __call__(self, cmd: StringObject) -> StringObject:
"""Executes the command in the shell and returns the result."""
return StringObject(value=subprocess.getoutput(cmd.value))
class PythonNativeBaseErrorObject(BaseException, PythonNativeObject):
pass
class PythonNativeSystemExitErrorObject(SystemExit, PythonNativeObject):
pass
class PythonNativeArithmeticErrorObject(ArithmeticError, PythonNativeObject):
pass
class PythonNativeAttributeErrorObject(AttributeError, PythonNativeObject):
pass
class PythonNativeLookupErrorObject(LookupError, PythonNativeObject):
pass
class PythonNativeOSErrorObject(OSError, PythonNativeObject):
pass
class PythonNativeRuntimeErrorObject(RuntimeError, PythonNativeObject):
pass
@dataclass
class StructInstanceObject(ExpressionObject):
environment: Environment = field(repr=False, kw_only=True)
def __eq__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks whether the attributes are similar."""
return BooleanObject(value=self.__dict__ == other)
def __ne__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks whether the attributes are different or not."""
return BooleanObject(value=self.__dict__ != other)
def __hash__(self) -> int:
"""Returns the object ID as a hash."""
return id(self)
def __getattr__(self, name: str) -> FarrObject:
"""Finds the value from the environment."""
return self.environment.locate(name)
class StatementObject(FarrObject):
pass
@dataclass
class NonPythonNativeObject(StatementObject):
environment: Optional[Environment] = field(
default=None, repr=False, kw_only=True
)
body: BlockNode = field(repr=False, kw_only=True)
@dataclass
class FunctionDefinitionObject(NonPythonNativeObject):
params: ItemizedExpressionNode = field(repr=False, kw_only=True)
def __eq__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks whether the attributes are similar."""
return BooleanObject(value=self.__dict__ == other)
def __ne__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks whether the attributes are different or not."""
return BooleanObject(value=self.__dict__ != other)
def __hash__(self) -> int:
"""Returns the object ID as a hash."""
return id(self)
@dataclass
class StructDefinitionObject(NonPythonNativeObject):
attributes: ItemizedExpressionNode = field(repr=False, kw_only=True)
def __eq__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks whether the attributes are similar."""
return BooleanObject(value=self.__dict__ == other)
def __ne__(self, other: FarrObject) -> BooleanObject: # type: ignore[override]
"""Checks whether the attributes are different or not."""
return BooleanObject(value=self.__dict__ != other)
def __hash__(self) -> int:
"""Returns the object ID as a hash."""
return id(self)
Of course, it is not necessary in this section to not divide the objects into two groups of expression and statement (it wasn't even necessary in the parser phase, but it was done for simplicity)...
Conclusion
If I want to design and implement a programming language again (seriously, not just to gain experience); with this project, I realized that I should spend more time on the design phase so that I don't get into ideological conflicts on the way...
See the project on GitHub, test it and also give feedback (if you like)!
0
Subscribe to my newsletter
Read articles from Artin Mohammadi directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Artin Mohammadi
Artin Mohammadi
A software developer who sometimes creates content — I'm very eager to get a deeper experience by implementing things from scratch.