Press ESC to close

PRIMARY RAC & STANDBY RAC DATAGUARD CONFIGURATON ORACLE19c & DATA GUARD BROKER SETTING

İki nodlu RAC primary database ve 2 nodlu RAC Standby Database üzerinde data guard konfigureasyonu yaplcak; Bunun için her iki veritabanında bire bir aynı versiyon ve patch seviyesinde olması gerekmektedir. RAC veritabanı kurulumu için; Standby veritabanımızın cluster yapılandırılması yapılmış, asm diskleri hazır ve rdbms yazılımı software only olarak kurulmuş halde olmalıdır.

https://www.farukcevik.com.tr/oracle-rac-kurulumu/

Creating a Physical Standby using RMAN Duplicate (RAC or Non-RAC) (Doc ID 1617946.1)

ROLE DB_NAME DB_UNIQUE_NAME Oracle Service Name Instances Hostnames
Primary frkdb frkdb frkdb frkdb1, frkdb2 dbnode1, dbnode2
Standby frkdb sfrkdb sfrkdb dgfrkdb1,dgfrkdb2 dgnode1, dgnode2

Data Guard için primary ve standby taraflarında dbname ler aynı db_unique nameler farklı olmak zorundadır.

Primary veritabanımız için;

[oracle@dbnode1 ~]$ srvctl config database -d frkdb
Database unique name: frkdb
Database name: frkdb
Oracle home: /u01/app/oracle/product/19.0.0.0/db
Oracle user: oracle
Spfile: +DATA/FRKDB/PARAMETERFILE/spfile.273.1085488901
Password file: +DATA/FRKDB/PASSWORD/pwdfrkdb.256.1085488289
Domain: farukcevik.com.tr
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: FRA,DATA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: oper
Database instances: frkdb1,frkdb2
Configured nodes: dbnode1,dbnode2
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed

Öncelikle primary veritabanımızın archivelog modda olduğundan ve Force logging enable olduğundan emin olalım.

SQL> select log_mode,database_role,open_mode from v$database;
LOG_MODE     DATABASE_ROLE    OPEN_MODE
------------ ---------------- --------------------
NOARCHIVELOG PRIMARY          READ WRITE

SQL> select force_logging from v$database;
FORCE_LOGGING
---------------------------------------
NO

Veritabanımızı arhivalog moda alalım ve force logging özelliğini aktif edeilm;

$ srvctl stop database -d frkdb
$ srvctl start database -d frkdb -o mount
$ sqlplus / as sysdba
SQL> alter database archivelog;
$ srvctl stop database -d frkdb
$ srvctl start database -d frkdb
$ sqlplus / as sysdba
SQL> alter database force logging;

Standby tarafında yeni geçici bir listener oluşturalım ki RMAN duplicate için ONS servisleri ile database bağlantısı kurabilelim, netca le 1525 portunu dinleneyen LISTENER_duplicate adında bir Listener oluşturuldu;

Primary tarafta parametrelerimiz aşağıdaki gibi düzenlendi;

SQL> alter system set log_archive_config='DG_CONFIG=(frkdb,sfrkdb)' scope=both sid='*';

SQL> alter system set log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=frkdb' scope= both sid='*';

SQL> alter system set log_archive_dest_2='SERVICE=sfrkdb VALID_FOR=(ONLINE_LOGFILES, PRIMARY_ROLE) DB_UNIQUE_NAME=sfrkdb' scope=both sid='*';

SQL> alter system set LOG_ARCHIVE_MAX_PROCESSES=8 scope=both sid='*' ;

SQL> alter system set FAL_SERVER=’frkdb,sfrkdb’ scope=both sid=’*’

--Eğer primary ve standby tarafında disk grouplarımız farklı ise bu yapılandırmayı yapmalıyız
SQL> alter system set DB_FILE_NAME_CONVERT='+DATA','+DATADG' scope=spfile sid='*'
SQL> alter system set LOG_FILE_NAME_CONVERT= '+FRA','+FRADG' scope=spfile sid='*' 

SQL> alter system set STANDBY_FILE_MANAGEMENT=AUTO scope=spfile sid='*'

SQL> alter system set LOG_ARCHIVE_DEST_STATE_1=ENABLE;

Standby için primary tarafta redo log file oluşturmalıyız, önerilen instance başına normalden bir fazla oluşturulmasıdır, ayrıca mevcut redo log dosyaları ile aynı boyutta olmalıdır.

Mevcut redo loglarımıza bakalım

SQL> select group#,thread#,bytes/1024/1024 MB from v$log;

    GROUP#    THREAD#         MB
---------- ---------- ----------
         1                 2                 200
         2                 2                 200
         3                 1                 200
         4                 1                 200

Stanby için oluşturalım;

SQL> ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 ('+FRA') SIZE 200M; 
SQL> ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 ('+FRA') SIZE 200M; 
SQL> ALTER DATABASE ADD STANDBY LOGFILE THREAD 1 ('+FRA') SIZE 200M; 
SQL> ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 ('+FRA') SIZE 200M; 
SQL> ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 ('+FRA') SIZE 200M; 
SQL> ALTER DATABASE ADD STANDBY LOGFILE THREAD 2 ('+FRA') SIZE 200M;

Tüm parametreler gözden geçirilmelidir, Data guard parametrelerimizi gözen geçirelim;

PARAMETER RAC PRIMARY DATABASE   RAC STANDBY DATABASE
DG_BROKER_CONFIG_FILE1 +DATA/FRKDB/dr1.dat   +DATA/SFRKDB/dr1.dat
DG_BROKER_CONFIG_FILE2 +FRA/FRKDB/dr2.dat   +FRA/SFRKDB/dr2.dat’
DB_BROKER_START true   true
LOG_ARCHIVE_CONFIG DG_CONFID=(frkdb,sfrkdb)   DG_CONFID=(sfrkdb,frkdb)
LOG_ARCHIVE_DEST1  LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,PRMARY_ROLE) DB_UNIQUE_NAME=FRKDB   LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,PRMARY_ROLE) DB_UNIQUE_NAME=SFRKDB
LOG_ARCHIVE_DEST_2 SERVICE=sfrkdb VALID_FOR (ONLNE_LOGFILES, PRIMARY_ROLE) DB_UNIQUE_NAME=sfrkdb   SERVICE=frkdb VALID_FOR (ONLNE_LOGFILES, PRIMARY_ROLE) DB_UNIQUE_NAME=frkdb
  DB_NAME frkdb   frkdb
  DB_UNIQUE_NAME frkdb     sfrkdb
  LOG_ARCHIVE_FORMAT %t_%s_%r.arc   %t_%s_%r.arc
  REMOTE_LOGIN_PASSWORDFILE EXCLUSIVE   EXCLUSIVE
  FAL_SERVER ‘dgnode1,dgnode2’     ‘dbnode1,dbnode2’
  FAL_CLIENT ‘dbnode1,dbnode2’   ‘dgnode1,dgnode2’
  DB_FILE_NAME_CONVERT ‘+DATA/SFRKDB/DATAFILE’, ‘+DATA/FRKDB/DATAFILE’   ‘+DATA/FRKDB/DATAFILE’, ‘+DATA/SFRKDB/DATAFILE’
LOG_FILE_NAME_CONVERT ‘+DATA/SFRKDB/ONLINELOG’, ‘+DATA/FRKDB/ ONLINELOG’, ‘+FRA/SFRKDB/ONLINELOG’, ‘+FRA/FRKDB/ONLINELOG’   ‘+DATA/FRKDB/ONLINELOG’, ‘+DATA/SFRKDB/ ONLINELOG’, ‘+FRA/FRKDB/ONLINELOG’, ‘+FRA/SFRKDB/ONLINELOG’
  STANDBY_FILE_MANAGEMENT AUTO   AUTO

Standby tarafta $DB_HOME/dbs dizini altında bir parameter file oluşturuyoruz;

$ cat /u01/app/oracle/product/19.0.0.0/db/dbs/initdgfrkdb1.ora

*.db_name='frkdb'
*.DB_UNIQUE_NAME='sfrkdb'
*.log_archive_config='DG_CONFIG=(sfrkdb,frkdb)'
*.log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=sfrkdb'
*.log_archive_dest_2='SERVICE=frkdb VALID_FOR=(ONLINE_LOGFILES, PRIMARY_ROLE) DB_UNIQUE_NAME=frkdb'
*.audit_file_dest='/u01/app/oracle/admin/sfrkdb/adump'
*.remote_login_passwordfile='exclusive'
*.db_recovery_file_dest='+FRA'
*.db_recovery_file_dest_size=8256m
*.db_create_file_dest='+DATA'
*.db_create_online_log_dest_1='+DATA'
*.db_create_online_log_dest_2='+FRA'

RMAN Duplicate Komutunun çalışabilmesi için Oracle Net servisinin ayarlanması gerekmektedir. Standby tarafında oluşturduğumuz LISTENER_duplicate ile auxiliary bağlantısı yapabilmek için listenera kayıt yapmalıyız, aşağıdaki kayıdı listener.ora dosyasına ekleyelim; ($DB_HOME/network/admin/listener.ora)

SID_LIST_LISTENER_duplicate =
  (SID_LIST =
   (SID_DESC =
    (SID_NAME = dgfrkdb1)
    (ORACLE_HOME = /u01/app/oracle/product/19.0.0.0/db)
    )
   )

TNS dosyasına tns alias giriyoruz; ($DB_HOME/network/admin/tnsnames.ora)

DUPDB =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = dgnode1)(PORT = 1525))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SID = dgfrkdb1) (UR=A)  )  )

Primary tarafta bir passwor dosyas oluşturalım ve standby tarafına, standby instance ismine göre isim değişikliği yaparak taşıyalım;

--Primary site
$ orapwd file='/u01/app/oracle/product/19.0.0.0/db/dbs/orapwfrkdb1'
$ scp /u01/app/oracle/product/19.0.0.0/db/dbs/orapwfrkdb1 oracle@dgnode1:/u01/app/oracle/product/19.0.0.0/db/dbs/orapwdgfrkdb1;

Stanby tarafta her iki node içinde aşağıdaki dizini oluşturalım;

$ mkdir –p /u01/app/oracle/admin/sfrkdb/adump

Tüm tnsnames.ora dosyalarına aşağıdaki kayıtları girelim.

Primary standby
FRKDB =  
(DESCRIPTION =    
(ADDRESS = (PROTOCOL = TCP)
(HOST = scan-frkdb.farukcevik.com.tr)
(PORT = 1521))    
(CONNECT_DATA =      
(SERVER = DEDICATED)      
(SERVICE_NAME = frkdb.farukcevik.com.tr)    
)  
)  

SFRKDB =  
(DESCRIPTION =    
(ADDRESS_LIST =      
(ADDRESS=
(PROTOCOL = TCP)
(HOST=scan-dg)(PORT = 1521))     )    
(CONNECT_DATA =      
(SERVER = DEDICATED)      
(SERVICE_NAME = sfrkdb)    
)  
)  

FRKDB_local_listener =  
(DESCRIPTION =    
(ADDRESS_LIST =      
(ADDRESS=(PROTOCOL=TCP)
(HOST=dbnode<?>-vip)
(PORT = 1521))     )   )


FRKDB =   (DESCRIPTION =    
(ADDRESS = (PROTOCOL = TCP)
(HOST = scan-frkdb.farukcevik.com.tr)
(PORT = 1521))    
(CONNECT_DATA =      
(SERVER = DEDICATED)      
(SERVICE_NAME = frkdb.farukcevik.com.tr)    
)  
)  

SFRKDB =  
(DESCRIPTION =    
(ADDRESS_LIST =      
(ADDRESS=(PROTOCOL = TCP)
(HOST=scan-dg)(PORT = 1521))    )    
(CONNECT_DATA =      
(SERVER = DEDICATED)      
(SERVICE_NAME = sfrkdb)    
)  
)    

SFRKDB_local_listener =  
(DESCRIPTION =    
(ADDRESS_LIST =      
(ADDRESS=(PROTOCOL = TCP)
(HOST=dgnode<?>-vip)(PORT = 1521))    
)  
)


Duplicate işlemi için RMAN işlemi,

[oracle@dbnode1 admin]$ rman target sys/Welcome_1@frkdb auxiliary sys/Welcome_1@dupdb

RMAN> duplicate target database for standby from active database nofilenamecheck;

veya aşağıdaki RMAN scriptini de çalşıtrabiliriz;

rman <<EOF
connect target sys/Welcome_1@frkdb;
connect auxiliary sys/Welcome_1@dupdp;
run {
duplicate target database for standby from active database nofilenamechcek
spfile
parameter_value_convert 'frkdb','sfrkdb'
set db_unique_name='sfrkdb'
set audit_file_dest='/u01/app/oracle/admin/sfrkdb/adump'
section size=5g;
}
EOF

Register olalım

RMAN> alter system register;

geçici olarak konumlandırdığımız LISTENER_duplicate i netca ile tekrar kaldırabiliriz.

Standby tarafta password dosyamızı ASM disklerine ekleyelim;

[oracle@dgnode1 ~]$ asmcmd
ASMCMD> cd DATA
ASMCMD> ls
SFRKDB/
ASMCMD> cd SFRKDB
ASMCMD> ls
CONTROLFILE/
DATAFILE/
ONLINELOG/
ASMCMD> mkdir PASSWORD
ASMCMD> pwcopy /u01/app/oracle/product/19.0.0.0/db/dbs/orapwdgfrkdb1 +DATA/SFRKDB/PASSWORD/pwsfrkdb

[oracle@dgnode1 ~]$ mv /u01/app/oracle/product/19.0.0.0/db/dbs/orapwdgfrkdb1 /u01/app/oracle/product/19.0.0.0/db/dbs/orapwold

RAC için konfigürasyonumuzu tamamlayalım;

parametre dosyamıza aşağıdaki satırları ekliyoruz;

Primary Standby
*.cluster_database=TRUE
frkdb2.instance_number=2
frkdb1.instance_number=1
frkdb2.thread=2
frkdb1.thread=1
frkdb2.undo_tablespace='UNDOTBS2'
frkdb1.undo_tablespace='UNDOTBS1'
……….

……….
*.cluster_database=TRUE
dgfrkdb2.instance_number=2
dgfrkdb1.instance_number=1
dgfrkdb2.thread=2
dgfrkdb1.thread=1
dgfrkdb2.undo_tablespace='UNDOTBS2'
dgfrkdb1.undo_tablespace='UNDOTBS1'
……….
……….

CRS a instanslarımız kaydetmeliyiz ki databaseimiz cluster olarak çalışabilsin;

$ srvctl add database -db sfrkdb -oraclehome /u01/app/oracle/product/19.0.0.0/db
$ srvctl add instance -db sfrkdb -instance dgfrkdb1 -node dgnode1
$ srvctl add instance -db sfrkdb -instance dgfrkdb2 -node dgnode2
$srvctl modify database -db sfrkdb -role physical_standby -spfile '+DATA/SFRKDB/spfileSFRKDB.ora' -pwfile '+DATA/SFRKDB/PASSWORD/pwsfrkdb'

$ srvctl start database -d sfrkdb -o mount
 
--Kontrol edelim

$ srvctl status database -d sfrkdb
Instance dgfrkdb1 is running on node dgnode1
Instance dgfrkdb2 is running on node dgnode2

Standby tarafında MRP (Managed Recovery Process) Prosesini çalıştıralım

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;

Kontrollerimizi yapalım; Primary tarafta rol kontrolü yapalım

SQL> SELECT DATABASE_ROLE, OPEN_MODE, SWITCHOVER_STATUS
 FROM V$DATABASE; 

DATABASE_ROLE    OPEN_MODE            SWITCHOVER_STATUS
---------------- -------------------- --------------------
PRIMARY          READ WRITE           TO STANDBY

Standby tarafta

SQL> SELECT DATABASE_ROLE, OPEN_MODE, SWITCHOVER_STATUS FROM V$DATABASE; 

DATABASE_ROLE    OPEN_MODE            SWITCHOVER_STATUS
---------------- -------------------- --------------------
PHYSICAL STANDBY MOUNTED              NOT ALLOWED

Primary tarafta

SQL> SELECT PROCESS, STATUS FROM V$MANAGED_STANDBY;

PROCESS   STATUS
--------- ------------
DGRD      ALLOCATED
DGRD      ALLOCATED
ARCH      CLOSING
ARCH      OPENING
ARCH      CONNECTED
ARCH      CONNECTED
ARCH      CONNECTED
ARCH      CLOSING
ARCH      CONNECTED
ARCH      CONNECTED
LNS       OPENING

PROCESS   STATUS
--------- ------------
DGRD      ALLOCATED
LNS       WRITING

13 rows selected.

Standby tarafta

SQL> SELECT PROCESS, STATUS FROM V$MANAGED_STANDBY;

PROCESS   STATUS
--------- ------------
ARCH      CONNECTED
DGRD      ALLOCATED
DGRD      ALLOCATED
ARCH      CONNECTED
ARCH      CONNECTED
ARCH      CONNECTED
RFS       IDLE
RFS       IDLE
RFS       IDLE
RFS       IDLE
RFS       IDLE

PROCESS   STATUS
--------- ------------
RFS       IDLE
MRP0      APPLYING_LOG

13 rows selected.

Primary ve Standby tarafında loglar sağlılık işleniyor görünüyor logswit ile bir test yapalım;

--Primary
SQL> select thread#,max(sequence#) from v$archived_log where archived='YES' group by thread#;
  THREAD# MAX(SEQUENCE#)
---------- --------------
         1             17
         2              6
--Standby
  THREAD# MAX(SEQUENCE#)
---------- --------------
         1             17
         2              6
--Primary
SQL> alter system switch logfile;

System altered.

SQL> select thread#,max(sequence#) from v$archived_log where archived='YES' group by thread#;

   THREAD# MAX(SEQUENCE#)
---------- --------------
         1             18
         2              6
--Standby Tarafta
SQL> select thread#,max(sequence#) from v$archived_log where archived='YES' group by thread#;

   THREAD# MAX(SEQUENCE#)
---------- --------------
         1             18
         2              6

Testini yaptık Data Guard sağlıklı çalışıyor.

Data Guard Broker Configuration;

Data Guard Broker ile ilgili mimari için aşağıdakiyazıya bakılabilir;

https://www.farukcevik.com.tr/oracle-data-guard-broker/

Hem primary hem de standby tarafta Data Guard Broker metadata doysalarını ayarlayarak enable edelim;

Primary

SQL> alter system set dg_broker_config_file1='+DATA/FRKDB/dr1.dat' scope=both;
SQL> alter system set dg_broker_config_file2='+FRA/FRKDB/dr2.dat' scope=both;
SQL> alter system set dg_broker_start=true scope=both;

Standby;

SQL> alter system set dg_broker_config_file1='+DATA/SFRKDB/dr1.dat' scope=both;
SQL> alter system set dg_broker_config_file2='+FRA/SFRKDB/dr2.dat' scope=both;
SQL> alter system set dg_broker_start=true scope=both;

NOTE: (MOS Note: 1617946.1) : Oracle 12.1.0.2 versiyonundan sonra Data Guard Broker konfigürasyonu için artık statik “_DGMRL” ihtiyaç yoktur.

Primary tarafa dgmgrl ile bağlanarak configürasyonumuzu yapalım;

$ dgmgrl sys/Welcome_1
DGMGRL> create configuration 'dg_config' as  primary database is 'frkdb' connect identifier is frkdb;

Configuration "dg_config" created with primary database "frkdb"

DGMGRL> add database 'sfrkdb' as connect identifier is sfrkdb;
Error: ORA-16698: member has a LOG_ARCHIVE_DEST_n parameter with SERVICE attribute set

Failed.

Bu hatayı almamak için Primary ve Standby tarafında LOG_ARCHIVE_DEST_2 parametresini kaldırıp dgmgrl konfigurasyonundan sonra tekrar eski halenie geitreceğiz.

SQL> show parameter log_archive_dest_2

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_2                   string      SERVICE=sfrkdb VALID_FOR=(ONLI
                                                 NE_LOGFILES, PRIMARY_ROLE) DB_
                                                 UNIQUE_NAME=sfrkdb



SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='' scope=both;

--STANDBY Side

SQL> show parameter log_archive_dest_2

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_2                   string      SERVICE=frkdb VALID_FOR=(ONLIN
                                                 E_LOGFILES, PRIMARY_ROLE) DB_U
                                                 NIQUE_NAME=frkdb


SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='' scope=both;


--PRIMARY tarafta yeniden DGMGRL konfigürasyonunu yapalım;



$ dgmgrl sys/Welcome_1
DGMGRL> create configuration 'dg_config' as  primary database is 'frkdb' connect identifier is frkdb;

Configuration "dg_config" created with primary database "frkdb"

DGMGRL> add database 'sfrkdb' as connect identifier is sfrkdb;
Database "sfrkdb" added

--PARAMETRELERIMIZ ESKİ HALE GETIREYII UNUTMUYORUZ!!!
--PRIMARY SITE

SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=sfrkdb LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=sfrkdb' scope=both sid='*';

--STANDBY SITE

SQL> ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=frkdb LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=frkdb' scope=both sid='*';

Tüm ayarlamalarımız tamam artık DGMGRL enable yapabiliriz;

DGMGRL> enable configuration;
Enabled.

Aşağıdaki komutla standby ve primary atarftaki tüm parametrelerimizi görebiliriz

DGMGRL> show database verbose sfrkdb
DGMGRL> show database verbose sfrkdb

Database - sfrkdb

  Role:               PHYSICAL STANDBY
  Intended State:     APPLY-ON
  Transport Lag:      0 seconds (computed 0 seconds ago)
  Apply Lag:          0 seconds (computed 0 seconds ago)
  Average Apply Rate: 43.00 KByte/s
  Active Apply Rate:  1.22 MByte/s
  Maximum Apply Rate: 11.69 MByte/s
  Real Time Query:    OFF
  Instance(s):
    dgfrkdb1 (apply instance)
    dgfrkdb2
Properties:
    DGConnectIdentifier             = 'sfrkdb'
--Uzayan bir liste var ...

DGMGRL> show database verbose frkdb

Database - frkdb

  Role:               PRIMARY
  Intended State:     TRANSPORT-ON
  Instance(s):
    frkdb1
    frkdb2

  Properties:
--uzayan bir liste

--Configürasyonu görelim

DGMGRL> show configuration

Configuration - dg_config

  Protection Mode: MaxPerformance
  Members:
  frkdb  - Primary database
    sfrkdb - Physical standby database

Fast-Start Failover:  Disabled

Configuration Status:
SUCCESS   (status updated 54 seconds ago)

Bir sonraki yazıda, bu yapılandırma üzerinden swichover, failover, otomotik failover kısımlarını inceleyeceğiz.

Bir yanıt yazın

E-posta adresiniz yayınlanmayacak. Gerekli alanlar * ile işaretlenmişlerdir