| Current File : //proc/self/root/lib/python3/dist-packages/certbot_apache/_internal/interfaces.py |
"""ParserNode interface for interacting with configuration tree.
General description
-------------------
The ParserNode interfaces are designed to be able to contain all the parsing logic,
while allowing their users to interact with the configuration tree in a Pythonic
and well structured manner.
The structure allows easy traversal of the tree of ParserNodes. Each ParserNode
stores a reference to its ancestor and immediate children, allowing the user to
traverse the tree using built in interface methods as well as accessing the interface
properties directly.
ParserNode interface implementation should stand between the actual underlying
parser functionality and the business logic within Configurator code, interfacing
with both. The ParserNode tree is a result of configuration parsing action.
ParserNode tree will be in charge of maintaining the parser state and hence the
abstract syntax tree (AST). Interactions between ParserNode tree and underlying
parser should involve only parsing the configuration files to this structure, and
writing it back to the filesystem - while preserving the format including whitespaces.
For some implementations (Apache for example) it's important to keep track of and
to use state information while parsing conditional blocks and directives. This
allows the implementation to set a flag to parts of the parsed configuration
structure as not being in effect in a case of unmatched conditional block. It's
important to store these blocks in the tree as well in order to not to conduct
destructive actions (failing to write back parts of the configuration) while writing
the AST back to the filesystem.
The ParserNode tree is in charge of maintaining the its own structure while every
child node fetched with find - methods or by iterating its list of children can be
changed in place. When making changes the affected nodes should be flagged as "dirty"
in order for the parser implementation to figure out the parts of the configuration
that need to be written back to disk during the save() operation.
Metadata
--------
The metadata holds all the implementation specific attributes of the ParserNodes -
things like the positional information related to the AST, file paths, whitespacing,
and any other information relevant to the underlying parser engine.
Access to the metadata should be handled by implementation specific methods, allowing
the Configurator functionality to access the underlying information where needed.
For some implementations the node can be initialized using the information carried
in metadata alone. This is useful especially when populating the ParserNode tree
while parsing the configuration.
Apache implementation
---------------------
The Apache implementation of ParserNode interface requires some implementation
specific functionalities that are not described by the interface itself.
Initialization
When the user of a ParserNode class is creating these objects, they must specify
the parameters as described in the documentation for the __init__ methods below.
When these objects are created internally, however, some parameters may not be
needed because (possibly more detailed) information is included in the metadata
parameter. In this case, implementations can deviate from the required parameters
from __init__, however, they should still behave the same when metadata is not
provided.
For consistency internally, if an argument is provided directly in the ParserNode
initialization parameters as well as within metadata it's recommended to establish
clear behavior around this scenario within the implementation.
Conditional blocks
Apache configuration can have conditional blocks, for example: <IfModule ...>,
resulting the directives and subblocks within it being either enabled or disabled.
While find_* interface methods allow including the disabled parts of the configuration
tree in searches a special care needs to be taken while parsing the structure in
order to reflect the active state of configuration.
Whitespaces
Each ParserNode object is responsible of storing its prepending whitespace characters
in order to be able to write the AST back to filesystem like it was, preserving the
format, this applies for parameters of BlockNode and DirectiveNode as well.
When parameters of ParserNode are changed, the pre-existing whitespaces in the
parameter sequence are discarded, as the general reason for storing them is to
maintain the ability to write the configuration back to filesystem exactly like
it was. This loses its meaning when we have to change the directives or blocks
parameters for other reasons.
Searches and matching
Apache configuration is largely case insensitive, so the Apache implementation of
ParserNode interface needs to provide the user means to match block and directive
names and parameters in case insensitive manner. This does not apply to everything
however, for example the parameters of a conditional statement may be case sensitive.
For this reason the internal representation of data should not ignore the case.
"""
import abc
from typing import Any
from typing import Dict
from typing import List
from typing import Optional
from typing import Tuple
from typing import TypeVar
GenericParserNode = TypeVar("GenericParserNode", bound="ParserNode")
class ParserNode(metaclass=abc.ABCMeta):
"""
ParserNode is the basic building block of the tree of such nodes,
representing the structure of the configuration. It is largely meant to keep
the structure information intact and idiomatically accessible.
The root node as well as the child nodes of it should be instances of ParserNode.
Nodes keep track of their differences to on-disk representation of configuration
by marking modified ParserNodes as dirty to enable partial write-to-disk for
different files in the configuration structure.
While for the most parts the usage and the child types are obvious, "include"-
and similar directives are an exception to this rule. This is because of the
nature of include directives - which unroll the contents of another file or
configuration block to their place. While we could unroll the included nodes
to the parent tree, it remains important to keep the context of include nodes
separate in order to write back the original configuration as it was.
For parsers that require the implementation to keep track of the whitespacing,
it's responsibility of each ParserNode object itself to store its prepending
whitespaces in order to be able to reconstruct the complete configuration file
as it was when originally read from the disk.
ParserNode objects should have the following attributes:
# Reference to ancestor node, or None if the node is the root node of the
# configuration tree.
ancestor: Optional[ParserNode]
# True if this node has been modified since last save.
dirty: bool
# Filepath of the file where the configuration element for this ParserNode
# object resides. For root node, the value for filepath is the httpd root
# configuration file. Filepath can be None if a configuration directive is
# defined in for example the httpd command line.
filepath: Optional[str]
# Metadata dictionary holds all the implementation specific key-value pairs
# for the ParserNode instance.
metadata: Dict[str, Any]
"""
ancestor: Optional["ParserNode"]
dirty: bool
filepath: Optional[str]
metadata: Dict[str, Any]
@abc.abstractmethod
def __init__(self, **kwargs: Any) -> None:
"""
Initializes the ParserNode instance, and sets the ParserNode specific
instance variables. This is not meant to be used directly, but through
specific classes implementing ParserNode interface.
:param ancestor: BlockNode ancestor for this CommentNode. Required.
:type ancestor: BlockNode or None
:param filepath: Filesystem path for the file where this CommentNode
does or should exist in the filesystem. Required.
:type filepath: str or None
:param dirty: Boolean flag for denoting if this CommentNode has been
created or changed after the last save. Default: False.
:type dirty: bool
:param metadata: Dictionary of metadata values for this ParserNode object.
Metadata information should be used only internally in the implementation.
Default: {}
:type metadata: dict
"""
@abc.abstractmethod
def save(self, msg: str) -> None:
"""
Save traverses the children, and attempts to write the AST to disk for
all the objects that are marked dirty. The actual operation of course
depends on the underlying implementation. save() shouldn't be called
from the Configurator outside of its designated save() method in order
to ensure that the Reverter checkpoints are created properly.
Note: this approach of keeping internal structure of the configuration
within the ParserNode tree does not represent the file inclusion structure
of actual configuration files that reside in the filesystem. To handle
file writes properly, the file specific temporary trees should be extracted
from the full ParserNode tree where necessary when writing to disk.
:param str msg: Message describing the reason for the save.
"""
@abc.abstractmethod
def find_ancestors(self: GenericParserNode, name: str) -> List[GenericParserNode]:
"""
Traverses the ancestor tree up, searching for BlockNodes with a specific
name.
:param str name: Name of the ancestor BlockNode to search for
:returns: A list of ancestor BlockNodes that match the name
:rtype: list of BlockNode
"""
class CommentNode(ParserNode, metaclass=abc.ABCMeta):
"""
CommentNode class is used for representation of comments within the parsed
configuration structure. Because of the nature of comments, it is not able
to have child nodes and hence it is always treated as a leaf node.
CommentNode stores its contents in class variable 'comment' and does not
have a specific name.
CommentNode objects should have the following attributes in addition to
the ones described in ParserNode:
# Contains the contents of the comment without the directive notation
# (typically # or /* ... */).
comment: str
"""
comment: str
@abc.abstractmethod
def __init__(self, **kwargs: Any) -> None:
"""
Initializes the CommentNode instance and sets its instance variables.
:param comment: Contents of the comment. Required.
:type comment: str
:param ancestor: BlockNode ancestor for this CommentNode. Required.
:type ancestor: BlockNode or None
:param filepath: Filesystem path for the file where this CommentNode
does or should exist in the filesystem. Required.
:type filepath: str or None
:param dirty: Boolean flag for denoting if this CommentNode has been
created or changed after the last save. Default: False.
:type dirty: bool
"""
super().__init__( # pragma: no cover
ancestor=kwargs['ancestor'],
dirty=kwargs.get('dirty', False),
filepath=kwargs['filepath'],
metadata=kwargs.get('metadata', {}),
)
class DirectiveNode(ParserNode, metaclass=abc.ABCMeta):
"""
DirectiveNode class represents a configuration directive within the configuration.
It can have zero or more parameters attached to it. Because of the nature of
single directives, it is not able to have child nodes and hence it is always
treated as a leaf node.
If a this directive was defined on the httpd command line, the ancestor instance
variable for this DirectiveNode should be None, and it should be inserted to the
beginning of root BlockNode children sequence.
DirectiveNode objects should have the following attributes in addition to
the ones described in ParserNode:
# True if this DirectiveNode is enabled and False if it is inside of an
# inactive conditional block.
enabled: bool
# Name, or key of the configuration directive. If BlockNode subclass of
# DirectiveNode is the root configuration node, the name should be None.
name: Optional[str]
# Tuple of parameters of this ParserNode object, excluding whitespaces.
parameters: Tuple[str, ...]
"""
enabled: bool
name: Optional[str]
parameters: Tuple[str, ...]
@abc.abstractmethod
def __init__(self, **kwargs: Any) -> None:
"""
Initializes the DirectiveNode instance and sets its instance variables.
:param name: Name or key of the DirectiveNode object. Required.
:type name: str or None
:param tuple parameters: Tuple of str parameters for this DirectiveNode.
Default: ().
:type parameters: tuple
:param ancestor: BlockNode ancestor for this DirectiveNode, or None for
root configuration node. Required.
:type ancestor: BlockNode or None
:param filepath: Filesystem path for the file where this DirectiveNode
does or should exist in the filesystem, or None for directives introduced
in the httpd command line. Required.
:type filepath: str or None
:param dirty: Boolean flag for denoting if this DirectiveNode has been
created or changed after the last save. Default: False.
:type dirty: bool
:param enabled: True if this DirectiveNode object is parsed in the active
configuration of the httpd. False if the DirectiveNode exists within a
unmatched conditional configuration block. Default: True.
:type enabled: bool
"""
super().__init__( # pragma: no cover
ancestor=kwargs['ancestor'],
dirty=kwargs.get('dirty', False),
filepath=kwargs['filepath'],
metadata=kwargs.get('metadata', {}),
)
@abc.abstractmethod
def set_parameters(self, parameters: List[str]) -> None:
"""
Sets the sequence of parameters for this ParserNode object without
whitespaces. While the whitespaces for parameters are discarded when using
this method, the whitespacing preceding the ParserNode itself should be
kept intact.
:param list parameters: sequence of parameters
"""
class BlockNode(DirectiveNode, metaclass=abc.ABCMeta):
"""
BlockNode class represents a block of nested configuration directives, comments
and other blocks as its children. A BlockNode can have zero or more parameters
attached to it.
Configuration blocks typically consist of one or more child nodes of all possible
types. Because of this, the BlockNode class has various discovery and structure
management methods.
Lists of parameters used as an optional argument for some of the methods should
be lists of strings that are applicable parameters for each specific BlockNode
or DirectiveNode type. As an example, for a following configuration example:
<VirtualHost *:80>
...
</VirtualHost>
The node type would be BlockNode, name would be 'VirtualHost' and its parameters
would be: ['*:80'].
While for the following example:
LoadModule alias_module /usr/lib/apache2/modules/mod_alias.so
The node type would be DirectiveNode, name would be 'LoadModule' and its
parameters would be: ['alias_module', '/usr/lib/apache2/modules/mod_alias.so']
The applicable parameters are dependent on the underlying configuration language
and its grammar.
BlockNode objects should have the following attributes in addition to
the ones described in DirectiveNode:
# Tuple of direct children of this BlockNode object. The order of children
# in this tuple retain the order of elements in the parsed configuration
# block.
children: Tuple[ParserNode, ...]
"""
children: Tuple[ParserNode, ...]
@abc.abstractmethod
def add_child_block(self, name: str, parameters: Optional[List[str]] = None,
position: Optional[int] = None) -> "BlockNode":
"""
Adds a new BlockNode child node with provided values and marks the callee
BlockNode dirty. This is used to add new children to the AST. The preceding
whitespaces should not be added based on the ancestor or siblings for the
newly created object. This is to match the current behavior of the legacy
parser implementation.
:param str name: The name of the child node to add
:param list parameters: list of parameters for the node
:param int position: Position in the list of children to add the new child
node to. Defaults to None, which appends the newly created node to the list.
If an integer is given, the child is inserted before that index in the
list similar to list().insert.
:returns: BlockNode instance of the created child block
"""
@abc.abstractmethod
def add_child_directive(self, name: str, parameters: Optional[List[str]] = None,
position: Optional[int] = None) -> DirectiveNode:
"""
Adds a new DirectiveNode child node with provided values and marks the
callee BlockNode dirty. This is used to add new children to the AST. The
preceding whitespaces should not be added based on the ancestor or siblings
for the newly created object. This is to match the current behavior of the
legacy parser implementation.
:param str name: The name of the child node to add
:param list parameters: list of parameters for the node
:param int position: Position in the list of children to add the new child
node to. Defaults to None, which appends the newly created node to the list.
If an integer is given, the child is inserted before that index in the
list similar to list().insert.
:returns: DirectiveNode instance of the created child directive
"""
@abc.abstractmethod
def add_child_comment(self, comment: str = "", position: Optional[int] = None) -> CommentNode:
"""
Adds a new CommentNode child node with provided value and marks the
callee BlockNode dirty. This is used to add new children to the AST. The
preceding whitespaces should not be added based on the ancestor or siblings
for the newly created object. This is to match the current behavior of the
legacy parser implementation.
:param str comment: Comment contents
:param int position: Position in the list of children to add the new child
node to. Defaults to None, which appends the newly created node to the list.
If an integer is given, the child is inserted before that index in the
list similar to list().insert.
:returns: CommentNode instance of the created child comment
"""
@abc.abstractmethod
def find_blocks(self, name: str, exclude: bool = True) -> List["BlockNode"]:
"""
Find a configuration block by name. This method walks the child tree of
ParserNodes under the instance it was called from. This way it is possible
to search for the whole configuration tree, when starting from root node or
to do a partial search when starting from a specified branch. The lookup
should be case insensitive.
:param str name: The name of the directive to search for
:param bool exclude: If the search results should exclude the contents of
ParserNode objects that reside within conditional blocks and because
of current state are not enabled.
:returns: A list of found BlockNode objects.
"""
@abc.abstractmethod
def find_directives(self, name: str, exclude: bool = True) -> List[DirectiveNode]:
"""
Find a directive by name. This method walks the child tree of ParserNodes
under the instance it was called from. This way it is possible to search
for the whole configuration tree, when starting from root node, or to do
a partial search when starting from a specified branch. The lookup should
be case insensitive.
:param str name: The name of the directive to search for
:param bool exclude: If the search results should exclude the contents of
ParserNode objects that reside within conditional blocks and because
of current state are not enabled.
:returns: A list of found DirectiveNode objects.
"""
@abc.abstractmethod
def find_comments(self, comment: str) -> List[CommentNode]:
"""
Find comments with value containing the search term.
This method walks the child tree of ParserNodes under the instance it was
called from. This way it is possible to search for the whole configuration
tree, when starting from root node, or to do a partial search when starting
from a specified branch. The lookup should be case sensitive.
:param str comment: The content of comment to search for
:returns: A list of found CommentNode objects.
"""
@abc.abstractmethod
def delete_child(self, child: ParserNode) -> None:
"""
Remove a specified child node from the list of children of the called
BlockNode object.
:param ParserNode child: Child ParserNode object to remove from the list
of children of the callee.
"""
@abc.abstractmethod
def unsaved_files(self) -> List[str]:
"""
Returns a list of file paths that have been changed since the last save
(or the initial configuration parse). The intended use for this method
is to tell the Reverter which files need to be included in a checkpoint.
This is typically called for the root of the ParserNode tree.
:returns: list of file paths of files that have been changed but not yet
saved to disk.
"""
@abc.abstractmethod
def parsed_paths(self) -> List[str]:
"""
Returns a list of file paths that have currently been parsed into the parser
tree. The returned list may include paths with wildcard characters, for
example: ['/etc/apache2/conf.d/*.load']
This is typically called on the root node of the ParserNode tree.
:returns: list of file paths of files that have been parsed
"""